-- Logs begin at Wed 2022-08-24 12:47:09 UTC, end at Wed 2022-08-24 13:59:17 UTC. -- Aug 24 13:17:02.769978 dv-cinder-779578-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 13:17:02.779992 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 13:17:02 2022] *** Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: nodename: dv-cinder-779578-4-iscsi Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: machine: x86_64 Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: clock source: unix Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: pcre jit disabled Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: detected number of CPU cores: 4 Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: current working directory: / Aug 24 13:17:02.782096 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: detected binary path: /usr/bin/uwsgi-core Aug 24 13:17:02.782731 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: your processes number limit is 47680 Aug 24 13:17:02.782731 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: your memory page size is 4096 bytes Aug 24 13:17:02.782731 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: detected max file descriptor number: 2048 Aug 24 13:17:02.782731 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: lock engine: pthread robust mutexes Aug 24 13:17:02.782731 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: thunder lock: enabled Aug 24 13:17:02.782731 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 13:17:02.782731 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 13:17:02.823214 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: Python main interpreter initialized at 0x558f1bbd99c0 Aug 24 13:17:02.823214 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: python threads support enabled Aug 24 13:17:02.823214 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: your server socket listen backlog is limited to 100 connections Aug 24 13:17:02.823214 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: your mercy for graceful operations on workers is 90 seconds Aug 24 13:17:02.823609 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 13:17:02.823684 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: *** Operational MODE: preforking *** Aug 24 13:17:02.823961 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: *** uWSGI is running in multiple interpreter mode *** Aug 24 13:17:02.823961 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: spawned uWSGI master process (pid: 81683) Aug 24 13:17:02.824209 dv-cinder-779578-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 13:17:02.828228 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: spawned uWSGI worker 1 (pid: 81685, cores: 1) Aug 24 13:17:02.828228 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: spawned uWSGI worker 2 (pid: 81686, cores: 1) Aug 24 13:17:02.828228 dv-cinder-779578-4-iscsi devstack@keystone.service[81683]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 13:17:07.243233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 13:17:07.243233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: from cryptography.utils import int_from_bytes Aug 24 13:17:07.244047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 13:17:07.244047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: from cryptography.utils import int_from_bytes Aug 24 13:17:07.244207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 13:17:07.244207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: from cryptography.utils import int_from_bytes Aug 24 13:17:07.244207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 13:17:07.244207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: from cryptography.utils import int_from_bytes Aug 24 13:17:07.299810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 13:17:07.300012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 13:17:07.300172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] command line args: [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 13:17:07.300470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 13:17:07.300606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 13:17:07.300694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ================================================================================ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 13:17:07.300807 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 13:17:07.300893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] admin_token = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.301004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] command line args: [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 13:17:07.301126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] config_dir = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.301275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 13:17:07.301358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.301456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ================================================================================ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 13:17:07.301539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] config_source = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.301653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] debug = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.301735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] admin_token = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.301857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] config_dir = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] config_source = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] debug = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.302892 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.303024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.303183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.303328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.303439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.303570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_config_append = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.303676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.303777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.303985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.304100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_dir = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.304242 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_config_append = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.304356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_file = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.304460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.304562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.304662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_dir = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.304763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_file = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.304873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.305947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.306059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.306059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.306286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.306445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.306561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.306674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.306786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.306786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.307932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.308051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.308163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.308305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.308429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] publish_errors = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.308541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.308658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.308836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.308943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] publish_errors = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.309060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.309179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.309414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.309527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.309656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.309776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] standard_threads = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.309902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.310022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.310136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.310281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.310397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] standard_threads = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.310597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.310742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.310924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_journal = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_json = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_stderr = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_journal = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] use_syslog = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_json = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.311970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_stderr = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.312083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.312292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] use_syslog = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.312417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.312537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 13:17:07.312651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.312769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.312883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.313940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.314095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.314218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.314332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.314456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.314599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.314711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.314823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.314935 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.315987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.316102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.316227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.316342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.316467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.316587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.316707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.316821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.316933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.317045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.317166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.317281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.317393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.317505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.317655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.317771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.317964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.318086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.318200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.318311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.318422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.318545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.318699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.318814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.318999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.319990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.320113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.320258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.320376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.320489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.320601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.320713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.320838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.320953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321065 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.321976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322188 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322809 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.322915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.323980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.324133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.324282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.324399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.324509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.324623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.324737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.324840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.324959 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.325081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.325185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.325297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.325422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.325533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.325655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.325776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.325890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326695 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.326932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.327055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.327171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.327323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.327445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.327560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.327674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.327795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.327911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.328978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.329096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.329284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.329402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.329402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.329605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.329723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.329844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.329968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.330083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.330196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.330309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.330309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.330531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.330657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.330787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.330914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.331951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.332065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.332272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.332391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.332505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.332618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.332717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.332812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.332934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.333861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.334083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.334739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.334868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.335990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.336103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.336242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.336360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.336474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.336588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.336702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.336815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.336936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.external = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.external = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.337996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.338121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.password = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.338121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] auth.token = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.338339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.338454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.338576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.338699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.338814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.password = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.338933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] auth.token = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.339980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.340096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.340270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.340389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.340504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.340619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.340742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.340867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.340983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.341098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.341212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.341328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.341452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.341568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.341689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.341806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.341927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.342995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.343976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.344082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.344197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.344301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.344399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.349809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.350481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.350564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.350625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.350684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.350790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.350896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.350960 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.351938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352892 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.352962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353253 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.353969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.354968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355261 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355882 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.355953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.356960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357186 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.357956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.358958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.359033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.359136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.359217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.359371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.359452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.359623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.359774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.359874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.360018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.360125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.360312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.360392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.360570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.360648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.360876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.360966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.361114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.361193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.361353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.361438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.361587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.361674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.361837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.361916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.362078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.362155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.362317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.362405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.362562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.362640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.362811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.362890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.363044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.363131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.363292 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.363371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.363531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.363609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.363765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.363853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.364987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.365070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.365235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.365322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.365514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.365601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.365757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.365863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.366019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.366090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.366256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.366335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.366504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.366583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.366747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.366831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.367974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.368051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.368235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.368318 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.368555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.368734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.368820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.368982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.369064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.369224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.369302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.369476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.369553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.369714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.369794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.369953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] role.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] role.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] role.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.370973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.371142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.371221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] role.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.371374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.371451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.371626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.371703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.371873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.371953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.372119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.372244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.372391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.372470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.372639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.372721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.372871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.372949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.373119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.373205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.373366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.373447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.373617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.373751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.373892 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.373999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.374105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.374351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.374457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.374585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.374689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.374836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.375028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.375141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.375270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.375377 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.375525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.375641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.375780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.375887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.376029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.376133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.376303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.377807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.377807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.378963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.379889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.380107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.380282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.380400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.380514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.380641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.380641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.380859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.380974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.381116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.381235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.381343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.381443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.381540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.381695 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.381831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.381946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.382962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.383077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.383192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.383307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.383434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.383552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.383666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.383781 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.383896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.384872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.385126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.385246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.385362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.385485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.385610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.385726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.385849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.385965 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.386080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.386194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.386308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.386423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.386615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.386735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.386850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.386964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.387078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.387191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.387313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.387431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.387545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.387659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.387774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.387887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.388990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.389105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81685) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 13:17:07.389221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.389221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.389221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 13:17:07.389221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81686) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 13:17:07.417795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.418025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.418297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.418583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.418825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.419054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.427180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.428625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.457482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.457708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.458548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.458787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.513336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.514780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.571730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.571959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.572895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.573138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.738233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.738452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.738538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.738687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.738772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.738913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.738998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.739140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.739233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.739351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.739438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.739570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.771220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.771391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.771480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.771558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.771733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.772370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.772502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.772624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.772755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.772948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.773041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.773118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.773191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.773264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.784511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.784678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.805713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.805952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.806173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.806397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.806498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.806618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.806700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.806835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.806916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.807053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81686) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.807136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.807315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.807524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.807734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81685) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 13:17:07.824724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.831051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.831182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 13:17:07.831287 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 13:17:07.831398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.831497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.831608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 13:17:07.831710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 13:17:07.831808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.831904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.832001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.832099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.832224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.832347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.832463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.832579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.836913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.837027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.837204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.837286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.837483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.837774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.837855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.838096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.844845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.845137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.847785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.848025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.848123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.848339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.848422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.848627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.848708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.848912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.848994 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.849185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.849302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.849447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.849527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.849788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.849881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.850043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.850352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.850432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.850685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.850764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.850959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.851038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.873632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.878605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.878696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.878696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.878830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.878830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.880026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.880341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.880556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.880637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.880848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.881801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.887625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.887916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.888392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.888688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.892243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.892542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.892828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.892980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.893117 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.893280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.893396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.893569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.893674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.893861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.893941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.894147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.894235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.894429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.894703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.894977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.912110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.912451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.913103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.913396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.916500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.917352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.919707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.920004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.920745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.921051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.924536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.925664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.927680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.929612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.931648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.931950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.932270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.936745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.949799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.950094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.950365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.950633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.951007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.951309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.951586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.951859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.957545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.958930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.959340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.959429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.960686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.960975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.962122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.962421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.962712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.963007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.963287 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.963641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.963941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.964264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.964568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.964852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.974541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.974854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.975279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.975745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.976036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.976349 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.976644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.976856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.977149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.977437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.977727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.978016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.978301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.978603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:07.992907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.994704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.995948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.997806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:07.998888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.000360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.000737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.001890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:08.002175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.002255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.002466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.003709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:08.003999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.004325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.009612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.011118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:08.011586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.013094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:08.014136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.014430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.014719 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.015008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.016134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.016456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.016847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.017150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.023559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:08.023796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.024080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.027211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:08.027500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.028000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.031454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81685) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:08.031754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.032038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.032354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.032633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.032919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.033195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.033476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.033753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.034029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.034310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.034572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.034846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.035115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81685) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.036099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81686) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 13:17:08.036424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.036714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.036999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.037576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.037720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.037947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.038353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.038804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.039041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.039321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.039608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.039882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.040162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81686) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 13:17:08.071985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558f1bbd99c0 pid: 81685 (default app) Aug 24 13:17:08.075585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a08e5b-2c36-4cc4-86c1-7c9c4b780cdc None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:08.075872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a08e5b-2c36-4cc4-86c1-7c9c4b780cdc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:08.076148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a08e5b-2c36-4cc4-86c1-7c9c4b780cdc None None] PATH_INFO: `/v3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:08.076669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558f1bbd99c0 pid: 81686 (default app) Aug 24 13:17:08.080318 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Wed Aug 24 13:17:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:15.650726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bf87f-1f09-4dcb-8452-23e15da90279 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:15.651279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bf87f-1f09-4dcb-8452-23e15da90279 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:15.651279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bf87f-1f09-4dcb-8452-23e15da90279 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:15.655570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1/2] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:15.662859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:15.663359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:15.663735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:15.733252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: INFO dbcounter [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] Registered counter for database keystone Aug 24 13:17:15.754512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG oslo_db.sqlalchemy.engines [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81685) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 13:17:15.761711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writer thread running {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 13:17:15.945671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG passlib.handlers.bcrypt [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81685) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 13:17:15.946202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG passlib.handlers.bcrypt [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81685) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 13:17:16.001961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:16.003568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:16.141381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 2/3] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 13:17:16.641533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bf87f-1f09-4dcb-8452-23e15da90279 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:16.711034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: INFO dbcounter [None req-388bf87f-1f09-4dcb-8452-23e15da90279 None None] Registered counter for database keystone Aug 24 13:17:16.730568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG oslo_db.sqlalchemy.engines [None req-388bf87f-1f09-4dcb-8452-23e15da90279 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81686) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 13:17:16.736867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writer thread running {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 13:17:16.858231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bf87f-1f09-4dcb-8452-23e15da90279 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:16.862475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e1830-1e3b-4a0d-b15a-9100a54065e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:16.863123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e1830-1e3b-4a0d-b15a-9100a54065e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:16.863404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e1830-1e3b-4a0d-b15a-9100a54065e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:16.863729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e1830-1e3b-4a0d-b15a-9100a54065e0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:16.872923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e1830-1e3b-4a0d-b15a-9100a54065e0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:16.929334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING keystone.common.rbac_enforcer.enforcer [None req-483e1830-1e3b-4a0d-b15a-9100a54065e0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 13:17:16.929583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e1830-1e3b-4a0d-b15a-9100a54065e0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:16.939944 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 2/4] 10.222.0.73 () {58 vars in 1324 bytes} [Wed Aug 24 13:17:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 13:17:16.946062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:16.961342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb54919-2686-4419-8cc6-055969124bc1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:16.965926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c1135-2510-4c84-9402-c4366be66340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:16.967052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1135-2510-4c84-9402-c4366be66340 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:16.967338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1135-2510-4c84-9402-c4366be66340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:16.967609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1135-2510-4c84-9402-c4366be66340 admin admin] PATH_INFO: `/v3/projects/1e28975466b54f229ea6496a2befdfd6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:16.969246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c1135-2510-4c84-9402-c4366be66340 admin admin] RBAC: Authorizing `identity:get_project(project_id=1e28975466b54f229ea6496a2befdfd6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:17.022124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c99c1135-2510-4c84-9402-c4366be66340 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 13:17:17.022549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c1135-2510-4c84-9402-c4366be66340 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:17.027266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 3/5] 10.222.0.73 () {56 vars in 1207 bytes} [Wed Aug 24 13:17:16 2022] GET /identity/v3/projects/1e28975466b54f229ea6496a2befdfd6 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:18.980201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f47a-fa6f-4a39-baee-d7c13b9a66da None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:18.980730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f47a-fa6f-4a39-baee-d7c13b9a66da None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:18.980846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f47a-fa6f-4a39-baee-d7c13b9a66da None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:18.984890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 3/6] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:18.990979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315589d0-c548-4405-8c03-d2ccc4eb5e22 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:18.991448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315589d0-c548-4405-8c03-d2ccc4eb5e22 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:18.991542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315589d0-c548-4405-8c03-d2ccc4eb5e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:19.011401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-315589d0-c548-4405-8c03-d2ccc4eb5e22 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:19.012881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-315589d0-c548-4405-8c03-d2ccc4eb5e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:19.020900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 4/7] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 13:17:19.291397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67f47a-fa6f-4a39-baee-d7c13b9a66da None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:19.304366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67f47a-fa6f-4a39-baee-d7c13b9a66da None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:19.308593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7373e2b-4d62-4e23-8729-2e75842619da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:19.309189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7373e2b-4d62-4e23-8729-2e75842619da admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:19.309471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7373e2b-4d62-4e23-8729-2e75842619da admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:19.309746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7373e2b-4d62-4e23-8729-2e75842619da admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:19.317487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7373e2b-4d62-4e23-8729-2e75842619da admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:19.319857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7373e2b-4d62-4e23-8729-2e75842619da admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:19.328735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 4/8] 10.222.0.73 () {58 vars in 1324 bytes} [Wed Aug 24 13:17:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:17:19.334972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315589d0-c548-4405-8c03-d2ccc4eb5e22 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:19.344228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315589d0-c548-4405-8c03-d2ccc4eb5e22 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:19.348654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fdc94-08f3-44b3-a095-c2ce0e52aec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:19.349285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fdc94-08f3-44b3-a095-c2ce0e52aec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:19.350067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fdc94-08f3-44b3-a095-c2ce0e52aec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:19.350352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fdc94-08f3-44b3-a095-c2ce0e52aec8 admin admin] PATH_INFO: `/v3/users/23e13f53885b4bcd974e51d34ba1099e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:19.351686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fdc94-08f3-44b3-a095-c2ce0e52aec8 admin admin] RBAC: Authorizing `identity:get_user(user_id=23e13f53885b4bcd974e51d34ba1099e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:19.353566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fdc94-08f3-44b3-a095-c2ce0e52aec8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:19.355119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 5/9] 10.222.0.73 () {56 vars in 1198 bytes} [Wed Aug 24 13:17:19 2022] GET /identity/v3/users/23e13f53885b4bcd974e51d34ba1099e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:21.385360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443fa79-c83f-40ec-a9bf-53b54e84e4cf None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:21.385886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443fa79-c83f-40ec-a9bf-53b54e84e4cf None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:21.385886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443fa79-c83f-40ec-a9bf-53b54e84e4cf None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:21.387230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 5/10] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:21.392736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e86d7-b332-4d41-89b7-ef6b15049ea7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:21.393156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e86d7-b332-4d41-89b7-ef6b15049ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:21.393292 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e86d7-b332-4d41-89b7-ef6b15049ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:21.400274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58903a36-efe4-4499-a75e-6a742c42198e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:21.400572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58903a36-efe4-4499-a75e-6a742c42198e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:21.400852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58903a36-efe4-4499-a75e-6a742c42198e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:21.402162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 6/11] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:21.407490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:21.407776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:21.408088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:21.412625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-710e86d7-b332-4d41-89b7-ef6b15049ea7 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:21.413817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-710e86d7-b332-4d41-89b7-ef6b15049ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:21.452902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG passlib.handlers.bcrypt [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81686) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 13:17:21.453332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG passlib.handlers.bcrypt [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81686) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 13:17:21.496003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 6/12] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:21.500109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:21.501748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:21.601763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 7/13] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:21.765473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e86d7-b332-4d41-89b7-ef6b15049ea7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:21.790345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:21.865701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:21.878239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e86d7-b332-4d41-89b7-ef6b15049ea7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:21.915996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c7654-6241-46e2-86de-c799e5afd8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:21.916667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c7654-6241-46e2-86de-c799e5afd8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:21.916953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c7654-6241-46e2-86de-c799e5afd8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:21.917229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c7654-6241-46e2-86de-c799e5afd8f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:21.931051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c7654-6241-46e2-86de-c799e5afd8f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-018c7654-6241-46e2-86de-c799e5afd8f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:21.939584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:21.940576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:21.941597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 7/14] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:21.946507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c7654-6241-46e2-86de-c799e5afd8f2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:21.978956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb8c7e-aa5f-49de-bd7f-73213e73e4a8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:21.981239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:22.016583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d9e9c-915a-486b-9966-2ef470551f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:22.017365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d9e9c-915a-486b-9966-2ef470551f17 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:22.017652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d9e9c-915a-486b-9966-2ef470551f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:22.017928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d9e9c-915a-486b-9966-2ef470551f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:22.039878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:22.058951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c7654-6241-46e2-86de-c799e5afd8f2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:22.085651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d9e9c-915a-486b-9966-2ef470551f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:22.087254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d9e9c-915a-486b-9966-2ef470551f17 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:22.094517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:22.095249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:22.095535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:22.095815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:22.096532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:22.096843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:22.098328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:22.111080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 8/15] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:22.117567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:22.207344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 8/16] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:17:22.213973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d9e9c-915a-486b-9966-2ef470551f17 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:22.216920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:22.240924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6be75-55a6-4a14-b613-36d5f33e5ccc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:22.277672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fdb3-cdb1-4e42-8ecc-6b40f1d36280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:22.278310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fdb3-cdb1-4e42-8ecc-6b40f1d36280 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:22.278594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fdb3-cdb1-4e42-8ecc-6b40f1d36280 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:22.278870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471fdb3-cdb1-4e42-8ecc-6b40f1d36280 None admin] PATH_INFO: `/v3/users/23e13f53885b4bcd974e51d34ba1099e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:22.280021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471fdb3-cdb1-4e42-8ecc-6b40f1d36280 None admin] RBAC: Authorizing `identity:get_user(user_id=23e13f53885b4bcd974e51d34ba1099e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:22.281522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471fdb3-cdb1-4e42-8ecc-6b40f1d36280 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:22.283121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 9/17] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:22 2022] GET /identity/v3/users/23e13f53885b4bcd974e51d34ba1099e => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:22.289320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fdb3-cdb1-4e42-8ecc-6b40f1d36280 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:22.304776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:22.315076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d9e9c-915a-486b-9966-2ef470551f17 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:22.358629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:22.359585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:22.359862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:22.360139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:22.387766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:22.389320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:22.398705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471fdb3-cdb1-4e42-8ecc-6b40f1d36280 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:17:22.408201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:22.409301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:17:22.410138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:17:22.410138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:17:22.410138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:22.410138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 9/18] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:17:22.415385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:22.437125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa8bff-f8d7-4bf0-904e-961d66bc2c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:22.437759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa8bff-f8d7-4bf0-904e-961d66bc2c91 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:22.438042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa8bff-f8d7-4bf0-904e-961d66bc2c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:22.438317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa8bff-f8d7-4bf0-904e-961d66bc2c91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:22.439468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa8bff-f8d7-4bf0-904e-961d66bc2c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:22.440965 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa8bff-f8d7-4bf0-904e-961d66bc2c91 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:22.442522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 10/19] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:22.449267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa8bff-f8d7-4bf0-904e-961d66bc2c91 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:22.474105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:22.518516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf9943-71c1-4939-bb62-2c0c7e42442d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:22.537302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:22.556867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e52c2-682c-4c34-a44a-d081193f58d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:22.557486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e52c2-682c-4c34-a44a-d081193f58d8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:22.557622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa8bff-f8d7-4bf0-904e-961d66bc2c91 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:22.557770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e52c2-682c-4c34-a44a-d081193f58d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:22.558045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e52c2-682c-4c34-a44a-d081193f58d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:22.558695 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e52c2-682c-4c34-a44a-d081193f58d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:22.559006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e52c2-682c-4c34-a44a-d081193f58d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:22.560465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e52c2-682c-4c34-a44a-d081193f58d8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:22.580099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 10/20] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:22.596396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894dd55-f8b2-4b2b-a89d-e5837ccf130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:22.597028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894dd55-f8b2-4b2b-a89d-e5837ccf130c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:22.597313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894dd55-f8b2-4b2b-a89d-e5837ccf130c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:22.597590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894dd55-f8b2-4b2b-a89d-e5837ccf130c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:22.598341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894dd55-f8b2-4b2b-a89d-e5837ccf130c None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.domain.id=default, user.id=23e13f53885b4bcd974e51d34ba1099e) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:22.598654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894dd55-f8b2-4b2b-a89d-e5837ccf130c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:22.601553 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894dd55-f8b2-4b2b-a89d-e5837ccf130c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:22.609312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 11/21] 10.222.0.73 () {56 vars in 1322 bytes} [Wed Aug 24 13:17:22 2022] GET /identity/v3/role_assignments?user.id=23e13f53885b4bcd974e51d34ba1099e&scope.domain.id=default&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:25.076150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6568950-61da-4a81-b931-6a9a37d84ce8 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.077180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6568950-61da-4a81-b931-6a9a37d84ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.077670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6568950-61da-4a81-b931-6a9a37d84ce8 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.079516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 11/22] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:25.084870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ed0a-cace-499d-8830-b47bd8d243e7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.085758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ed0a-cace-499d-8830-b47bd8d243e7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.086246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ed0a-cace-499d-8830-b47bd8d243e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.113214 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3aa1ed0a-cace-499d-8830-b47bd8d243e7 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:25.115295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3aa1ed0a-cace-499d-8830-b47bd8d243e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:25.198950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa786c2-0b44-4e96-83bb-0572e615bd31 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.199286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa786c2-0b44-4e96-83bb-0572e615bd31 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.199509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa786c2-0b44-4e96-83bb-0572e615bd31 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.201070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 12/23] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:25.206006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 12/24] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:25.216751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e648ea-2d96-400e-a993-a0b1086410e6 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.216751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e648ea-2d96-400e-a993-a0b1086410e6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.217017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e648ea-2d96-400e-a993-a0b1086410e6 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.218562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 13/25] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:25.219077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33599938-a40b-4358-aced-d7daef37e71a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.219636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33599938-a40b-4358-aced-d7daef37e71a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.220101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33599938-a40b-4358-aced-d7daef37e71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.225541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10802-1a42-4b5c-bae0-361e531fde45 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.225541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10802-1a42-4b5c-bae0-361e531fde45 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.225541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa10802-1a42-4b5c-bae0-361e531fde45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.248406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-33599938-a40b-4358-aced-d7daef37e71a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:25.249941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-33599938-a40b-4358-aced-d7daef37e71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:25.256539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6aa10802-1a42-4b5c-bae0-361e531fde45 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:25.257728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6aa10802-1a42-4b5c-bae0-361e531fde45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:25.362588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 13/26] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:25.387612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 14/27] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:25.678558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33599938-a40b-4358-aced-d7daef37e71a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:25.682673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:25.821004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33599938-a40b-4358-aced-d7daef37e71a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:25.836646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f416bd-fa2a-4d9e-a360-56c16fde4633 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.837267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f416bd-fa2a-4d9e-a360-56c16fde4633 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.837737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f416bd-fa2a-4d9e-a360-56c16fde4633 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.839311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 15/28] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:25.850165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968301d-c9aa-4c27-818d-c9969d980f82 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.850743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968301d-c9aa-4c27-818d-c9969d980f82 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.851214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968301d-c9aa-4c27-818d-c9969d980f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.881477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:25.882124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:25.882403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:25.882675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:25.884113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d968301d-c9aa-4c27-818d-c9969d980f82 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:25.885335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d968301d-c9aa-4c27-818d-c9969d980f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:25.900978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:25.903148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:17:25.923525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:25.924483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:17:25.925254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:17:25.925254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:17:25.925254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:25.927775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 14/29] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:25.928988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:25.947579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:25.979131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 16/30] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:25.984414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968301d-c9aa-4c27-818d-c9969d980f82 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:26.000628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:26.053724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4eb25f-c706-4cf4-b1bd-e5d2787efc28 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:26.102604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce73692-053d-4f7a-8d9a-fd10ae50c3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:26.104219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce73692-053d-4f7a-8d9a-fd10ae50c3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.105369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce73692-053d-4f7a-8d9a-fd10ae50c3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.105976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce73692-053d-4f7a-8d9a-fd10ae50c3f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.129958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce73692-053d-4f7a-8d9a-fd10ae50c3f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-6ce73692-053d-4f7a-8d9a-fd10ae50c3f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:26.137937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:26.138973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:26.139892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:26.144647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 15/31] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:17:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:26.147793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce73692-053d-4f7a-8d9a-fd10ae50c3f0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:26.232000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968301d-c9aa-4c27-818d-c9969d980f82 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:26.301084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce73692-053d-4f7a-8d9a-fd10ae50c3f0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:26.320606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:26.334231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bca504-b3e1-43e8-a0c0-988a65081061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:26.335407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca504-b3e1-43e8-a0c0-988a65081061 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.335745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca504-b3e1-43e8-a0c0-988a65081061 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.336125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bca504-b3e1-43e8-a0c0-988a65081061 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.337296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bca504-b3e1-43e8-a0c0-988a65081061 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:26.339094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bca504-b3e1-43e8-a0c0-988a65081061 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:26.342111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:26.342893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.343386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.343795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.344644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:26.345095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:26.347101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:26.371637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 16/32] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:26.371637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:26.488341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23765e-b831-4ca3-93f7-72daf16c8270 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:26.524021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:26.524657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.524947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.525226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.525914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:26.526220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:26.527608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:26.535988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 17/33] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:26.537419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:26.554593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 17/34] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 13:17:25 2022] POST /identity/v3/roles => generated 219 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:26.555554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bca504-b3e1-43e8-a0c0-988a65081061 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:26.582968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:26.596794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:26.655845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b8735-95d2-4f86-b63c-5fc7f85a3323 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:26.672677 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bca504-b3e1-43e8-a0c0-988a65081061 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:26.699807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b9298-363d-47ce-ac2f-37561169fc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:26.700798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9298-363d-47ce-ac2f-37561169fc9e None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.701542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9298-363d-47ce-ac2f-37561169fc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.701542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b9298-363d-47ce-ac2f-37561169fc9e None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.702192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b9298-363d-47ce-ac2f-37561169fc9e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:26.703590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b9298-363d-47ce-ac2f-37561169fc9e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:26.713840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0503987-01d9-4a7a-83bc-777f32cb9d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:26.714450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0503987-01d9-4a7a-83bc-777f32cb9d13 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.714734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0503987-01d9-4a7a-83bc-777f32cb9d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.715012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0503987-01d9-4a7a-83bc-777f32cb9d13 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.715920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0503987-01d9-4a7a-83bc-777f32cb9d13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:26.717561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0503987-01d9-4a7a-83bc-777f32cb9d13 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:26.807581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 18/35] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:17:26 2022] POST /identity/v3/projects => generated 301 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:26.808974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef2ca3-54d4-4e5b-b491-bbff3d141b21 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.809281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef2ca3-54d4-4e5b-b491-bbff3d141b21 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.809542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef2ca3-54d4-4e5b-b491-bbff3d141b21 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.811326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 19/36] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:26.818532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9212924-5097-4b87-a04b-5b7ee2cac22e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.818532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9212924-5097-4b87-a04b-5b7ee2cac22e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.818532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9212924-5097-4b87-a04b-5b7ee2cac22e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.818532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 20/37] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:26.818532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9212924-5097-4b87-a04b-5b7ee2cac22e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:26.889878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 18/38] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 13:17:26 2022] POST /identity/v3/roles => generated 225 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:26.891636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c5200-709c-4f2e-8a61-a07ede64fff0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:26.891943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c5200-709c-4f2e-8a61-a07ede64fff0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:26.896787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c5200-709c-4f2e-8a61-a07ede64fff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:26.925462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-365c5200-709c-4f2e-8a61-a07ede64fff0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:26.927845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-365c5200-709c-4f2e-8a61-a07ede64fff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:26.954146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:26.974582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9212924-5097-4b87-a04b-5b7ee2cac22e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:27.027519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c756e-fee7-4b9a-862b-1de157f0f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:27.028209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c756e-fee7-4b9a-862b-1de157f0f57d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:27.028504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c756e-fee7-4b9a-862b-1de157f0f57d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:27.028794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c756e-fee7-4b9a-862b-1de157f0f57d None admin] PATH_INFO: `/v3/users/23e13f53885b4bcd974e51d34ba1099e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:27.030294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c756e-fee7-4b9a-862b-1de157f0f57d None admin] RBAC: Authorizing `identity:get_user(user_id=23e13f53885b4bcd974e51d34ba1099e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:27.031800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c756e-fee7-4b9a-862b-1de157f0f57d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:27.033866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 21/39] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:26 2022] GET /identity/v3/users/23e13f53885b4bcd974e51d34ba1099e => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:27.035164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe7e1f-7e50-4a8a-997d-053098551b9b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:27.035445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe7e1f-7e50-4a8a-997d-053098551b9b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:27.035703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe7e1f-7e50-4a8a-997d-053098551b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:27.060350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-28fe7e1f-7e50-4a8a-997d-053098551b9b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:27.061945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-28fe7e1f-7e50-4a8a-997d-053098551b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:27.062978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 19/40] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:27.066157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c5200-709c-4f2e-8a61-a07ede64fff0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:27.116286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:27.149602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 22/41] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:27.322317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c5200-709c-4f2e-8a61-a07ede64fff0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:27.351946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe7e1f-7e50-4a8a-997d-053098551b9b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:27.361010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b164a-4f1b-4169-b044-b2411adb61eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:27.361650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b164a-4f1b-4169-b044-b2411adb61eb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:27.361966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b164a-4f1b-4169-b044-b2411adb61eb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:27.362225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b164a-4f1b-4169-b044-b2411adb61eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:27.364037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b164a-4f1b-4169-b044-b2411adb61eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:27.366352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b164a-4f1b-4169-b044-b2411adb61eb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:27.367963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 20/42] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:27.373498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b164a-4f1b-4169-b044-b2411adb61eb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:27.449744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:27.471172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe7e1f-7e50-4a8a-997d-053098551b9b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:27.482070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:27.488315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b164a-4f1b-4169-b044-b2411adb61eb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:27.506991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8361d11-74b0-44db-b3bd-83c4c5ff9dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:27.512339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8361d11-74b0-44db-b3bd-83c4c5ff9dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:27.512339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8361d11-74b0-44db-b3bd-83c4c5ff9dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:27.512339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8361d11-74b0-44db-b3bd-83c4c5ff9dd7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:27.512339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8361d11-74b0-44db-b3bd-83c4c5ff9dd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:27.512339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8361d11-74b0-44db-b3bd-83c4c5ff9dd7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:27.512700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 23/43] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:27.517266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8361d11-74b0-44db-b3bd-83c4c5ff9dd7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:27.524860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8464e-3aeb-4252-8626-0acab27aa469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:27.525534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8464e-3aeb-4252-8626-0acab27aa469 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:27.525818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8464e-3aeb-4252-8626-0acab27aa469 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:27.526093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8464e-3aeb-4252-8626-0acab27aa469 None admin] PATH_INFO: `/v3/domains/default/users/23e13f53885b4bcd974e51d34ba1099e/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:27.528556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8464e-3aeb-4252-8626-0acab27aa469 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=23e13f53885b4bcd974e51d34ba1099e, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:27.530683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8464e-3aeb-4252-8626-0acab27aa469 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:27.627537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8361d11-74b0-44db-b3bd-83c4c5ff9dd7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:27.629024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 21/44] 10.222.0.73 () {58 vars in 1361 bytes} [Wed Aug 24 13:17:27 2022] PUT /identity/v3/domains/default/users/23e13f53885b4bcd974e51d34ba1099e/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:17:27.629885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8464e-3aeb-4252-8626-0acab27aa469 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:27.679646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770d3ae-74b3-481e-ae66-75ea38a33c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:27.680332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770d3ae-74b3-481e-ae66-75ea38a33c7c None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:27.680754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770d3ae-74b3-481e-ae66-75ea38a33c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:27.681063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770d3ae-74b3-481e-ae66-75ea38a33c7c None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:27.681783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770d3ae-74b3-481e-ae66-75ea38a33c7c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:27.683275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770d3ae-74b3-481e-ae66-75ea38a33c7c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:27.759619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8464e-3aeb-4252-8626-0acab27aa469 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:27.761746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 24/45] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 13:17:27 2022] POST /identity/v3/roles => generated 223 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:27.800481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e9175-b719-4432-b9cd-e82048179237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:27.801119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e9175-b719-4432-b9cd-e82048179237 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:27.801403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e9175-b719-4432-b9cd-e82048179237 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:27.801700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e9175-b719-4432-b9cd-e82048179237 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:27.803093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e9175-b719-4432-b9cd-e82048179237 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:27.804634 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e9175-b719-4432-b9cd-e82048179237 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:27.909363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 22/46] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:17:27 2022] POST /identity/v3/projects => generated 312 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:29.625739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd9e8b-18cd-402c-a8d9-5d884d327e6c None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:29.626479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd9e8b-18cd-402c-a8d9-5d884d327e6c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:29.626479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd9e8b-18cd-402c-a8d9-5d884d327e6c None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:29.627644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 25/47] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:29.634195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7291ccb-9f94-4f20-967a-a8d5b4f9db61 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:29.634852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7291ccb-9f94-4f20-967a-a8d5b4f9db61 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:29.635380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7291ccb-9f94-4f20-967a-a8d5b4f9db61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:29.666496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c7291ccb-9f94-4f20-967a-a8d5b4f9db61 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:29.668684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c7291ccb-9f94-4f20-967a-a8d5b4f9db61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:29.673025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:29.778969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 23/48] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:29.918807 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f3350-b459-4578-8588-d7bf3b972457 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:29.918998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f3350-b459-4578-8588-d7bf3b972457 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:29.919284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f3350-b459-4578-8588-d7bf3b972457 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:29.920681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 26/49] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:29.927175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c0474-0169-4244-b6fe-fa0d58bbc393 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:29.927472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c0474-0169-4244-b6fe-fa0d58bbc393 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:29.927752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c0474-0169-4244-b6fe-fa0d58bbc393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:29.946660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a30c0474-0169-4244-b6fe-fa0d58bbc393 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:29.948608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a30c0474-0169-4244-b6fe-fa0d58bbc393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:30.028751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 24/50] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:30.154377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f3350-b459-4578-8588-d7bf3b972457 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:30.181540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:30.275927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f3350-b459-4578-8588-d7bf3b972457 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:30.315555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52f1cf-5b96-4db8-b61b-2d451d55ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:30.316226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52f1cf-5b96-4db8-b61b-2d451d55ed86 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:30.316519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52f1cf-5b96-4db8-b61b-2d451d55ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:30.316799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52f1cf-5b96-4db8-b61b-2d451d55ed86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:30.323770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c0474-0169-4244-b6fe-fa0d58bbc393 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:30.331237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52f1cf-5b96-4db8-b61b-2d451d55ed86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-0c52f1cf-5b96-4db8-b61b-2d451d55ed86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:30.337785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:30.338970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:30.339883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 27/51] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:17:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:30.345294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52f1cf-5b96-4db8-b61b-2d451d55ed86 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:30.405157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:30.449561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:30.454677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c0474-0169-4244-b6fe-fa0d58bbc393 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:30.475066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52f1cf-5b96-4db8-b61b-2d451d55ed86 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:30.490532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100be35-caf3-48eb-b6d1-ea9f806b6137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:30.491161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100be35-caf3-48eb-b6d1-ea9f806b6137 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:30.491440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100be35-caf3-48eb-b6d1-ea9f806b6137 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:30.491709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100be35-caf3-48eb-b6d1-ea9f806b6137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:30.492832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100be35-caf3-48eb-b6d1-ea9f806b6137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:30.494282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100be35-caf3-48eb-b6d1-ea9f806b6137 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:30.495852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 25/52] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:30.501454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100be35-caf3-48eb-b6d1-ea9f806b6137 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:30.510949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:30.511370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:30.511651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:30.511922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:30.512636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:30.512940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:30.514356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:30.527554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 28/53] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:30.533149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:30.604273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100be35-caf3-48eb-b6d1-ea9f806b6137 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:30.637642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdfd5b-ae27-4203-8a3a-b96f2d853c7c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:30.639407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d7fd0-9bb2-4693-906b-aaf3f533897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:30.640019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d7fd0-9bb2-4693-906b-aaf3f533897e None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:30.640330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d7fd0-9bb2-4693-906b-aaf3f533897e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:30.640606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d7fd0-9bb2-4693-906b-aaf3f533897e None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:30.641251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d7fd0-9bb2-4693-906b-aaf3f533897e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:30.642696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91d7fd0-9bb2-4693-906b-aaf3f533897e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:30.648119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:30.672077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2582e-a7df-4d1a-8d3d-876b1e57a65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:30.673000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2582e-a7df-4d1a-8d3d-876b1e57a65c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:30.673285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2582e-a7df-4d1a-8d3d-876b1e57a65c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:30.673562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2582e-a7df-4d1a-8d3d-876b1e57a65c None admin] PATH_INFO: `/v3/users/23e13f53885b4bcd974e51d34ba1099e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:30.674710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2582e-a7df-4d1a-8d3d-876b1e57a65c None admin] RBAC: Authorizing `identity:get_user(user_id=23e13f53885b4bcd974e51d34ba1099e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:30.676200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2582e-a7df-4d1a-8d3d-876b1e57a65c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:30.678440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 29/54] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:30 2022] GET /identity/v3/users/23e13f53885b4bcd974e51d34ba1099e => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:30.684172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2582e-a7df-4d1a-8d3d-876b1e57a65c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:30.687842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:30.752602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 26/55] 10.222.0.73 () {60 vars in 1140 bytes} [Wed Aug 24 13:17:30 2022] POST /identity/v3/projects => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:17:30.789718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2582e-a7df-4d1a-8d3d-876b1e57a65c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:30.828590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8cbea-cbd5-4755-b5b6-4e4676dfb178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:30.829245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8cbea-cbd5-4755-b5b6-4e4676dfb178 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:30.829534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8cbea-cbd5-4755-b5b6-4e4676dfb178 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:30.829813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8cbea-cbd5-4755-b5b6-4e4676dfb178 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:30.831132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8cbea-cbd5-4755-b5b6-4e4676dfb178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:30.832683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8cbea-cbd5-4755-b5b6-4e4676dfb178 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:30.835059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 30/56] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:30.840906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d7fd0-9bb2-4693-906b-aaf3f533897e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:30.949824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d7fd0-9bb2-4693-906b-aaf3f533897e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:30.984358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcfd3c8-95a3-427f-92c1-8c58cfd40df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:30.984971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfd3c8-95a3-427f-92c1-8c58cfd40df8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:30.985257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfd3c8-95a3-427f-92c1-8c58cfd40df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:30.985794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfd3c8-95a3-427f-92c1-8c58cfd40df8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:30.986542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfd3c8-95a3-427f-92c1-8c58cfd40df8 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.domain.id=default, user.id=23e13f53885b4bcd974e51d34ba1099e) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:30.986855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfd3c8-95a3-427f-92c1-8c58cfd40df8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:30.988324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcfd3c8-95a3-427f-92c1-8c58cfd40df8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:30.995407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 27/57] 10.222.0.73 () {56 vars in 1322 bytes} [Wed Aug 24 13:17:30 2022] GET /identity/v3/role_assignments?user.id=23e13f53885b4bcd974e51d34ba1099e&scope.domain.id=default&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:32.765650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958e54-848a-4210-928a-801ac40ed0cb None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:32.766118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958e54-848a-4210-928a-801ac40ed0cb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:32.766221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958e54-848a-4210-928a-801ac40ed0cb None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:32.767565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 31/58] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:32.772984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8fd11-95ad-49be-98aa-265894059277 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:32.773431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8fd11-95ad-49be-98aa-265894059277 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:32.773546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8fd11-95ad-49be-98aa-265894059277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:32.791598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f9f8fd11-95ad-49be-98aa-265894059277 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:32.792843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f9f8fd11-95ad-49be-98aa-265894059277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:32.842470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:32.862376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 28/59] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:33.131126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59958e54-848a-4210-928a-801ac40ed0cb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:33.205715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:33.237943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59958e54-848a-4210-928a-801ac40ed0cb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:33.280932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bab048-401c-4b95-a6e3-6bfd39d9089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:33.280932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bab048-401c-4b95-a6e3-6bfd39d9089c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:33.280932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bab048-401c-4b95-a6e3-6bfd39d9089c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:33.280932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bab048-401c-4b95-a6e3-6bfd39d9089c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:33.281322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bab048-401c-4b95-a6e3-6bfd39d9089c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:33.281322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bab048-401c-4b95-a6e3-6bfd39d9089c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:33.281322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 32/60] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:33.283048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8fd11-95ad-49be-98aa-265894059277 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:33.380827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8fd11-95ad-49be-98aa-265894059277 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:33.414418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148da67-1edd-4fc4-8a33-04a71d7de262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:33.414939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148da67-1edd-4fc4-8a33-04a71d7de262 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:33.415219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148da67-1edd-4fc4-8a33-04a71d7de262 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:33.415489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148da67-1edd-4fc4-8a33-04a71d7de262 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:33.416126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148da67-1edd-4fc4-8a33-04a71d7de262 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:33.417596 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148da67-1edd-4fc4-8a33-04a71d7de262 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:33.487408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 29/61] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:17:33 2022] POST /identity/v3/users => generated 275 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:36.651738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe3e0f-405c-4180-866c-5b0d68e9bb86 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:36.652305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe3e0f-405c-4180-866c-5b0d68e9bb86 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:36.652305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe3e0f-405c-4180-866c-5b0d68e9bb86 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:36.653947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 33/62] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:36.659256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0100-a0be-4f49-a65f-8b0b7f999dc9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:36.659697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0100-a0be-4f49-a65f-8b0b7f999dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:36.659892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c0100-a0be-4f49-a65f-8b0b7f999dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:36.689818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75322c-601a-44ac-b170-6f9769523c56 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:36.690166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75322c-601a-44ac-b170-6f9769523c56 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:36.690449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75322c-601a-44ac-b170-6f9769523c56 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:36.691857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 34/63] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:36.696228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e15c0100-a0be-4f49-a65f-8b0b7f999dc9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:36.697540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e15c0100-a0be-4f49-a65f-8b0b7f999dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:36.705827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b1849-d6fb-45ad-bf01-25f4441e4972 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:36.706112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b1849-d6fb-45ad-bf01-25f4441e4972 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:36.706388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b1849-d6fb-45ad-bf01-25f4441e4972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:36.738142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-314b1849-d6fb-45ad-bf01-25f4441e4972 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:36.739885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-314b1849-d6fb-45ad-bf01-25f4441e4972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:36.772572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:36.877796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 30/64] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:36.957500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 35/65] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:37.018733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f7df6-fa73-4702-99d2-fbaea47cd8a3 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.019021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f7df6-fa73-4702-99d2-fbaea47cd8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.019297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f7df6-fa73-4702-99d2-fbaea47cd8a3 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.020912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 31/66] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:37.026289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba131a-3761-4b7f-8069-d5df2bb07ffe None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.026577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba131a-3761-4b7f-8069-d5df2bb07ffe None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.026852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba131a-3761-4b7f-8069-d5df2bb07ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.053301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ebba131a-3761-4b7f-8069-d5df2bb07ffe None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:37.054512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ebba131a-3761-4b7f-8069-d5df2bb07ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:37.138610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:37.140157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3df68-7b3a-49c9-bf6a-1e591a0d705d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.140463 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3df68-7b3a-49c9-bf6a-1e591a0d705d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.140739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3df68-7b3a-49c9-bf6a-1e591a0d705d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.142225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 32/67] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:37.147049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 36/68] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:37.152529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef574-3107-4b25-8d72-e358fd954f68 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.162639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef574-3107-4b25-8d72-e358fd954f68 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.162639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ef574-3107-4b25-8d72-e358fd954f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.176212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b70ef574-3107-4b25-8d72-e358fd954f68 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:37.178063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b70ef574-3107-4b25-8d72-e358fd954f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:37.255807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 33/69] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:37.261346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba131a-3761-4b7f-8069-d5df2bb07ffe None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:37.267062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df6c9-10c7-4b14-848f-eef248b5ab0f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.267352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df6c9-10c7-4b14-848f-eef248b5ab0f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.267633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415df6c9-10c7-4b14-848f-eef248b5ab0f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.269013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 34/70] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:37.276264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2585b1-2e8f-4e47-bd1b-cbd326b37490 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.276555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2585b1-2e8f-4e47-bd1b-cbd326b37490 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.276829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2585b1-2e8f-4e47-bd1b-cbd326b37490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.300577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9d2585b1-2e8f-4e47-bd1b-cbd326b37490 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:37.302420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9d2585b1-2e8f-4e47-bd1b-cbd326b37490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:37.381999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba131a-3761-4b7f-8069-d5df2bb07ffe None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:37.392560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 35/71] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 13:17:37.393991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2585b1-2e8f-4e47-bd1b-cbd326b37490 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:37.420297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6c1f-268e-435b-9de1-05f67118408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:37.420934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6c1f-268e-435b-9de1-05f67118408b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.421213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6c1f-268e-435b-9de1-05f67118408b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.421485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c6c1f-268e-435b-9de1-05f67118408b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.432426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:37.433901 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c6c1f-268e-435b-9de1-05f67118408b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7d6c6c1f-268e-435b-9de1-05f67118408b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:37.438912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:37.439836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:37.441222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 37/72] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:37.446057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6c1f-268e-435b-9de1-05f67118408b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:37.517743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2585b1-2e8f-4e47-bd1b-cbd326b37490 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:37.523097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:37.559728 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed48f2-edd0-4b38-bb29-e3a35b3237f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:37.560384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed48f2-edd0-4b38-bb29-e3a35b3237f4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.560666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed48f2-edd0-4b38-bb29-e3a35b3237f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.560940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed48f2-edd0-4b38-bb29-e3a35b3237f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.572448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed48f2-edd0-4b38-bb29-e3a35b3237f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-14ed48f2-edd0-4b38-bb29-e3a35b3237f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:37.577337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:37.578182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:37.579335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 36/73] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:37.581921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c6c1f-268e-435b-9de1-05f67118408b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:37.584672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed48f2-edd0-4b38-bb29-e3a35b3237f4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:37.623949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:37.624613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.624897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.625182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.625911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:37.626161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:37.627555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:37.637282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 38/74] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:37.638188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:37.684712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:37.697561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed48f2-edd0-4b38-bb29-e3a35b3237f4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:37.732747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01832be-377f-4187-8db8-1bc2bc843ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:37.733387 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01832be-377f-4187-8db8-1bc2bc843ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.733674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01832be-377f-4187-8db8-1bc2bc843ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.733951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01832be-377f-4187-8db8-1bc2bc843ff6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.746655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01832be-377f-4187-8db8-1bc2bc843ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:37.749070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e809afa-9c37-4cbb-9900-6a2deb9f120f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a01832be-377f-4187-8db8-1bc2bc843ff6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:37.750099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:37.750973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:37.751714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 37/75] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:37.753037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01832be-377f-4187-8db8-1bc2bc843ff6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:37.779168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:37.791310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:37.795531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.795531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.795531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.795531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:37.795531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:37.795531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:37.804142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 39/76] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:37.805002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:37.859607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01832be-377f-4187-8db8-1bc2bc843ff6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:37.894221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a717f7c-97e8-4ef1-849a-9f3a5873a320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:37.894857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a717f7c-97e8-4ef1-849a-9f3a5873a320 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.895140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a717f7c-97e8-4ef1-849a-9f3a5873a320 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.895647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a717f7c-97e8-4ef1-849a-9f3a5873a320 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.906141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a717f7c-97e8-4ef1-849a-9f3a5873a320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:37.910793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a03bd-be01-4be0-adb7-ea13b47ff461 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4a717f7c-97e8-4ef1-849a-9f3a5873a320 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:37.912462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:37.913271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:37.914042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 38/77] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:37.914667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a717f7c-97e8-4ef1-849a-9f3a5873a320 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:37.926486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:37.948988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e6105-54de-4e9b-ad9e-8fa0c31cfa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:37.949616 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e6105-54de-4e9b-ad9e-8fa0c31cfa93 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:37.949952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e6105-54de-4e9b-ad9e-8fa0c31cfa93 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:37.950295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e6105-54de-4e9b-ad9e-8fa0c31cfa93 None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:37.965869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e6105-54de-4e9b-ad9e-8fa0c31cfa93 None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:37.967330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e6105-54de-4e9b-ad9e-8fa0c31cfa93 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:37.969007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 40/78] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:37.971238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e6105-54de-4e9b-ad9e-8fa0c31cfa93 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:38.021038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a717f7c-97e8-4ef1-849a-9f3a5873a320 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.027967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:38.055247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cee596-69d8-44b8-aae5-ad272b747474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.055809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cee596-69d8-44b8-aae5-ad272b747474 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.056092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cee596-69d8-44b8-aae5-ad272b747474 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.056394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cee596-69d8-44b8-aae5-ad272b747474 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.057517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cee596-69d8-44b8-aae5-ad272b747474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.058978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cee596-69d8-44b8-aae5-ad272b747474 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.060780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 39/79] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:38.061391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cee596-69d8-44b8-aae5-ad272b747474 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:38.089829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e6105-54de-4e9b-ad9e-8fa0c31cfa93 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.125082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.125699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.125984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.126264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.126938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:38.127245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.128709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.136948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 41/80] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:17:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:38.137496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:38.137757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:38.162863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cee596-69d8-44b8-aae5-ad272b747474 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.199377 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19e70e-8fe2-4869-90bf-78772f656fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.200017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e70e-8fe2-4869-90bf-78772f656fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.200333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e70e-8fe2-4869-90bf-78772f656fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.200610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19e70e-8fe2-4869-90bf-78772f656fb2 None admin] PATH_INFO: `/v3/users/23e13f53885b4bcd974e51d34ba1099e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.201743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19e70e-8fe2-4869-90bf-78772f656fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=23e13f53885b4bcd974e51d34ba1099e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.203189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19e70e-8fe2-4869-90bf-78772f656fb2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.210754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 40/81] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:38 2022] GET /identity/v3/users/23e13f53885b4bcd974e51d34ba1099e => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:38.210754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19e70e-8fe2-4869-90bf-78772f656fb2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:38.248923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4790ed3-ded6-4e6a-90ac-583d451a51b1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.251240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:38.292149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.292816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.293104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.293383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.294700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:38.295292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.298360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.308304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 42/82] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:38.308956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:38.326319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19e70e-8fe2-4869-90bf-78772f656fb2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.362455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9868af4-4611-48c3-abb6-75f2c658383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.363074 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868af4-4611-48c3-abb6-75f2c658383f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.363364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868af4-4611-48c3-abb6-75f2c658383f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.363644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868af4-4611-48c3-abb6-75f2c658383f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.386002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:38.427786 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a85c7-e5c6-4692-bae5-171afa785887 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.446832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9868af4-4611-48c3-abb6-75f2c658383f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.448376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9868af4-4611-48c3-abb6-75f2c658383f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.463016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fecc3e-3a1a-406b-89e7-fcc2c09a744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.463634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fecc3e-3a1a-406b-89e7-fcc2c09a744d None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.463911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fecc3e-3a1a-406b-89e7-fcc2c09a744d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.464209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fecc3e-3a1a-406b-89e7-fcc2c09a744d None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.464869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fecc3e-3a1a-406b-89e7-fcc2c09a744d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.466310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fecc3e-3a1a-406b-89e7-fcc2c09a744d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.550964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 41/83] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:38.552673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9868af4-4611-48c3-abb6-75f2c658383f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:38.554605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 43/84] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:17:38 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:38.555430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fecc3e-3a1a-406b-89e7-fcc2c09a744d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:38.588099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:38.618523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:38.685448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fecc3e-3a1a-406b-89e7-fcc2c09a744d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.687335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9868af4-4611-48c3-abb6-75f2c658383f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.732136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670daa7-2ee7-4736-af6c-0f40d9eeed09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.732714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228fd3-8140-43e4-884a-70739439ba02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.733329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670daa7-2ee7-4736-af6c-0f40d9eeed09 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.733432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228fd3-8140-43e4-884a-70739439ba02 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.733625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228fd3-8140-43e4-884a-70739439ba02 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.733902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228fd3-8140-43e4-884a-70739439ba02 None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.734097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670daa7-2ee7-4736-af6c-0f40d9eeed09 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.734520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670daa7-2ee7-4736-af6c-0f40d9eeed09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.735782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf228fd3-8140-43e4-884a-70739439ba02 None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.737537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf228fd3-8140-43e4-884a-70739439ba02 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.740244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 42/85] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:38 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:38.741071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228fd3-8140-43e4-884a-70739439ba02 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:38.844797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670daa7-2ee7-4736-af6c-0f40d9eeed09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.846251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670daa7-2ee7-4736-af6c-0f40d9eeed09 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.885754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:38.885935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228fd3-8140-43e4-884a-70739439ba02 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:38.906383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:38.943477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21095c9-9ebe-4f03-a7cc-a38c742c14ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:38.944128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21095c9-9ebe-4f03-a7cc-a38c742c14ba None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:38.944459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21095c9-9ebe-4f03-a7cc-a38c742c14ba None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:38.944739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21095c9-9ebe-4f03-a7cc-a38c742c14ba None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:38.945964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21095c9-9ebe-4f03-a7cc-a38c742c14ba None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:38.948501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21095c9-9ebe-4f03-a7cc-a38c742c14ba None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:38.950533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 43/86] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:38 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:38.953566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21095c9-9ebe-4f03-a7cc-a38c742c14ba None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:39.000012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 44/87] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:39.001595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670daa7-2ee7-4736-af6c-0f40d9eeed09 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:39.072558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21095c9-9ebe-4f03-a7cc-a38c742c14ba None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:39.135565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670daa7-2ee7-4736-af6c-0f40d9eeed09 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:39.165763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:39.166771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd4efa-77e5-49f6-88c5-6b84e3ce8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:39.168262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd4efa-77e5-49f6-88c5-6b84e3ce8afd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:39.168763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd4efa-77e5-49f6-88c5-6b84e3ce8afd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:39.169225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd4efa-77e5-49f6-88c5-6b84e3ce8afd None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:39.171268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd4efa-77e5-49f6-88c5-6b84e3ce8afd None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:39.173554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd4efa-77e5-49f6-88c5-6b84e3ce8afd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:39.180285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 44/88] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:38 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:39.180441 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaca11-b526-4cb9-94ff-da81d7eecd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:39.181524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaca11-b526-4cb9-94ff-da81d7eecd59 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:39.181923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd4efa-77e5-49f6-88c5-6b84e3ce8afd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:39.182161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaca11-b526-4cb9-94ff-da81d7eecd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:39.184142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaca11-b526-4cb9-94ff-da81d7eecd59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:39.247041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:39.308077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccaca11-b526-4cb9-94ff-da81d7eecd59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:39.309630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccaca11-b526-4cb9-94ff-da81d7eecd59 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:39.311563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd4efa-77e5-49f6-88c5-6b84e3ce8afd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:39.361013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb138eab-2ad2-4300-ba9e-4b7bfdee8db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:39.361634 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb138eab-2ad2-4300-ba9e-4b7bfdee8db9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:39.361915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb138eab-2ad2-4300-ba9e-4b7bfdee8db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:39.362190 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb138eab-2ad2-4300-ba9e-4b7bfdee8db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:39.433619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 45/89] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:39.434809 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaca11-b526-4cb9-94ff-da81d7eecd59 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:39.463184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:39.467277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb138eab-2ad2-4300-ba9e-4b7bfdee8db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:39.469163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb138eab-2ad2-4300-ba9e-4b7bfdee8db9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:39.524986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:39.567976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaca11-b526-4cb9-94ff-da81d7eecd59 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:39.597949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 45/90] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:39.599754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb138eab-2ad2-4300-ba9e-4b7bfdee8db9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:39.613861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c660187-82a7-40bf-8c9e-bf85b4262005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:39.615402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c660187-82a7-40bf-8c9e-bf85b4262005 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:39.615793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c660187-82a7-40bf-8c9e-bf85b4262005 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:39.616204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c660187-82a7-40bf-8c9e-bf85b4262005 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:39.617629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c660187-82a7-40bf-8c9e-bf85b4262005 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:39.619315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c660187-82a7-40bf-8c9e-bf85b4262005 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:39.621566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 46/91] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:39 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:39.622610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c660187-82a7-40bf-8c9e-bf85b4262005 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:39.710576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb138eab-2ad2-4300-ba9e-4b7bfdee8db9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:39.730088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:39.742781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c660187-82a7-40bf-8c9e-bf85b4262005 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:39.749822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:39.750449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:39.750737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:39.751012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:39.751755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] RBAC: Adding query filter params (role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=822c049cabcb4edb8210aa9424fbb06a, user.id=07197b17f7014c6192df2c1a77382f9b) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:39.752065 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:39.754006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:39.760845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 46/92] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:39 2022] GET /identity/v3/role_assignments?user.id=07197b17f7014c6192df2c1a77382f9b&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:39.761794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:39.778306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:39.792681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1528bf-86c6-48d7-9b11-799b1cb38221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:39.793686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1528bf-86c6-48d7-9b11-799b1cb38221 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:39.794171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1528bf-86c6-48d7-9b11-799b1cb38221 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:39.794713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1528bf-86c6-48d7-9b11-799b1cb38221 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:39.796197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1528bf-86c6-48d7-9b11-799b1cb38221 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:39.798239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1528bf-86c6-48d7-9b11-799b1cb38221 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:39.800467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 47/93] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:39 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:39.801489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1528bf-86c6-48d7-9b11-799b1cb38221 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:39.958077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcb118-a666-4b7f-942a-bba4f121bc78 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:40.002077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1528bf-86c6-48d7-9b11-799b1cb38221 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:40.008919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbc6cd-f08c-44ac-a4ef-1b5e2d525cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:40.009808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbc6cd-f08c-44ac-a4ef-1b5e2d525cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:40.010300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbc6cd-f08c-44ac-a4ef-1b5e2d525cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:40.011821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbc6cd-f08c-44ac-a4ef-1b5e2d525cc7 None admin] PATH_INFO: `/v3/projects/c648c3e2b08c45a588fe2c99a61f92f5` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:40.013336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbc6cd-f08c-44ac-a4ef-1b5e2d525cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=c648c3e2b08c45a588fe2c99a61f92f5)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:40.015159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbc6cd-f08c-44ac-a4ef-1b5e2d525cc7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:40.017162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 47/94] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:39 2022] GET /identity/v3/projects/c648c3e2b08c45a588fe2c99a61f92f5 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:40.018446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbc6cd-f08c-44ac-a4ef-1b5e2d525cc7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:40.055421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:40.056036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:40.056343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:40.056617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:40.057406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=822c049cabcb4edb8210aa9424fbb06a, user.id=23e13f53885b4bcd974e51d34ba1099e) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:40.057715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:40.059236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:40.081747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 48/95] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:39 2022] GET /identity/v3/role_assignments?user.id=23e13f53885b4bcd974e51d34ba1099e&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:40.082601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:40.139281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:40.176731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:40.200172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbc6cd-f08c-44ac-a4ef-1b5e2d525cc7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:40.248222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d202a3-d044-4918-babe-c9e5c7451f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:40.248859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d202a3-d044-4918-babe-c9e5c7451f6a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:40.249142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d202a3-d044-4918-babe-c9e5c7451f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:40.250490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d202a3-d044-4918-babe-c9e5c7451f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:40.251258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d202a3-d044-4918-babe-c9e5c7451f6a None admin] RBAC: Adding query filter params (role.id=47878794de344232a226a95ce5771504, scope.project.id=822c049cabcb4edb8210aa9424fbb06a, user.id=07197b17f7014c6192df2c1a77382f9b) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:40.251571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d202a3-d044-4918-babe-c9e5c7451f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:40.253118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d202a3-d044-4918-babe-c9e5c7451f6a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:40.265027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 48/96] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:40 2022] GET /identity/v3/role_assignments?user.id=07197b17f7014c6192df2c1a77382f9b&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=47878794de344232a226a95ce5771504 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:40.340689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112c5be-db83-429c-ac46-c860e2df0955 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:40.405609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49945e39-8743-432a-9c46-e90e2b6a9392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:40.406251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49945e39-8743-432a-9c46-e90e2b6a9392 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:40.406536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49945e39-8743-432a-9c46-e90e2b6a9392 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:40.406813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49945e39-8743-432a-9c46-e90e2b6a9392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:40.407561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49945e39-8743-432a-9c46-e90e2b6a9392 None admin] RBAC: Adding query filter params (role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=c648c3e2b08c45a588fe2c99a61f92f5, user.id=07197b17f7014c6192df2c1a77382f9b) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:40.407875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49945e39-8743-432a-9c46-e90e2b6a9392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:40.409426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49945e39-8743-432a-9c46-e90e2b6a9392 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:40.420975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 49/97] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:40 2022] GET /identity/v3/role_assignments?user.id=07197b17f7014c6192df2c1a77382f9b&scope.project.id=c648c3e2b08c45a588fe2c99a61f92f5&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 259 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:41.023891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84136b-0410-4176-a16f-6107b044afb4 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:41.024939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84136b-0410-4176-a16f-6107b044afb4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:41.025431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84136b-0410-4176-a16f-6107b044afb4 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:41.026976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 49/98] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:41.036847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfafe0d-5675-47d0-b66f-cefee1425a8c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:41.037749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfafe0d-5675-47d0-b66f-cefee1425a8c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:41.038217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfafe0d-5675-47d0-b66f-cefee1425a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:41.049412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb27d34-13d0-4260-abcc-293ddbff0e57 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:41.049978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb27d34-13d0-4260-abcc-293ddbff0e57 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:41.050421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb27d34-13d0-4260-abcc-293ddbff0e57 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:41.052637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 50/99] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:41.059206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c22007-9550-420f-bada-dea03806879e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:41.059772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c22007-9550-420f-bada-dea03806879e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:41.060299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c22007-9550-420f-bada-dea03806879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:41.080746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5cfafe0d-5675-47d0-b66f-cefee1425a8c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:41.082227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5cfafe0d-5675-47d0-b66f-cefee1425a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:41.095561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-29c22007-9550-420f-bada-dea03806879e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:41.097303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-29c22007-9550-420f-bada-dea03806879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:41.242579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 50/100] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:41.258699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 51/101] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:41.684955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfafe0d-5675-47d0-b66f-cefee1425a8c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:41.709126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:41.791140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c22007-9550-420f-bada-dea03806879e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:41.820865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfafe0d-5675-47d0-b66f-cefee1425a8c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:41.877875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e69cd0-767c-45bb-b1e3-377714ea940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:41.878533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e69cd0-767c-45bb-b1e3-377714ea940b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:41.878824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e69cd0-767c-45bb-b1e3-377714ea940b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:41.879101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e69cd0-767c-45bb-b1e3-377714ea940b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:41.880990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e69cd0-767c-45bb-b1e3-377714ea940b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:41.882712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e69cd0-767c-45bb-b1e3-377714ea940b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:41.884388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 51/102] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:41.892216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e69cd0-767c-45bb-b1e3-377714ea940b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:42.032212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:42.032903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:42.074568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e69cd0-767c-45bb-b1e3-377714ea940b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:42.102644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c22007-9550-420f-bada-dea03806879e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:42.158647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d9ec-ead0-4270-899e-18f45274b857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:42.159310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d9ec-ead0-4270-899e-18f45274b857 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.159595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d9ec-ead0-4270-899e-18f45274b857 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.160033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d9ec-ead0-4270-899e-18f45274b857 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.160819 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d9ec-ead0-4270-899e-18f45274b857 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:42.164604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d9ec-ead0-4270-899e-18f45274b857 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:42.205557 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da211f-ad5d-4b3a-a5ef-0073b3b11f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:42.206214 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da211f-ad5d-4b3a-a5ef-0073b3b11f22 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.206502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da211f-ad5d-4b3a-a5ef-0073b3b11f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.206785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da211f-ad5d-4b3a-a5ef-0073b3b11f22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.232433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da211f-ad5d-4b3a-a5ef-0073b3b11f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-31da211f-ad5d-4b3a-a5ef-0073b3b11f22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:42.237274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:42.238978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:42.241161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 52/103] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:42.248477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da211f-ad5d-4b3a-a5ef-0073b3b11f22 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:42.270884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 52/104] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:17:41 2022] POST /identity/v3/projects => generated 302 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:17:42.407874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da211f-ad5d-4b3a-a5ef-0073b3b11f22 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:42.445509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf2dd-432d-4e3b-b027-56876458f78e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.446330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf2dd-432d-4e3b-b027-56876458f78e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.446913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdf2dd-432d-4e3b-b027-56876458f78e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.448536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 53/105] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:42.455018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28985c-6ebd-468d-978b-8cb226f4bab5 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.455636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28985c-6ebd-468d-978b-8cb226f4bab5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.456213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28985c-6ebd-468d-978b-8cb226f4bab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.456564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:42.457197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.457496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.458345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.459264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:42.459697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:42.461276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:42.463085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:42.477356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 53/106] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:42.484432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:42.490910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ce28985c-6ebd-468d-978b-8cb226f4bab5 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:42.490910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ce28985c-6ebd-468d-978b-8cb226f4bab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:42.660434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee7698b-4b77-4d4f-9b5f-890a54578054 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:42.666977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 54/107] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:42.704895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf35748-02ef-4026-89e8-9a39cc02ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:42.705552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35748-02ef-4026-89e8-9a39cc02ce8b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.705838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35748-02ef-4026-89e8-9a39cc02ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.706144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35748-02ef-4026-89e8-9a39cc02ce8b None admin] PATH_INFO: `/v3/users/27352662136f47c8845c3766faf50315` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.725016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf35748-02ef-4026-89e8-9a39cc02ce8b None admin] RBAC: Authorizing `identity:get_user(user_id=27352662136f47c8845c3766faf50315)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:42.726928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf35748-02ef-4026-89e8-9a39cc02ce8b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:42.729452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 54/108] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:42 2022] GET /identity/v3/users/27352662136f47c8845c3766faf50315 => generated 289 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:42.735424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28985c-6ebd-468d-978b-8cb226f4bab5 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:42.775653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7395bb-b969-4e18-a0cd-6f5fa8afcdfd None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.776005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7395bb-b969-4e18-a0cd-6f5fa8afcdfd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.776232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7395bb-b969-4e18-a0cd-6f5fa8afcdfd None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.777649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 55/109] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:42.804477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1551d5-9b8f-43fd-9b83-a5a3e80d88fb None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.804779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1551d5-9b8f-43fd-9b83-a5a3e80d88fb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.805037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1551d5-9b8f-43fd-9b83-a5a3e80d88fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.843041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0d1551d5-9b8f-43fd-9b83-a5a3e80d88fb None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:42.844298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0d1551d5-9b8f-43fd-9b83-a5a3e80d88fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:42.867494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:42.916450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28985c-6ebd-468d-978b-8cb226f4bab5 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:42.956598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:42.963594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747050b-68c5-499f-a099-33002bf66e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:42.965171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747050b-68c5-499f-a099-33002bf66e80 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:42.965171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747050b-68c5-499f-a099-33002bf66e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:42.965171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747050b-68c5-499f-a099-33002bf66e80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:42.977959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 56/110] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:43.022429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1551d5-9b8f-43fd-9b83-a5a3e80d88fb None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:43.056933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747050b-68c5-499f-a099-33002bf66e80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:43.058489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747050b-68c5-499f-a099-33002bf66e80 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:43.168634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:43.235340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1551d5-9b8f-43fd-9b83-a5a3e80d88fb None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:43.241494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 55/111] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:43.247611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747050b-68c5-499f-a099-33002bf66e80 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:43.295574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1f9d-4901-4bbe-9421-084d7f5cff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:43.296236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1f9d-4901-4bbe-9421-084d7f5cff15 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.296520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1f9d-4901-4bbe-9421-084d7f5cff15 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.296792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1f9d-4901-4bbe-9421-084d7f5cff15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.316409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e1f9d-4901-4bbe-9421-084d7f5cff15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a57e1f9d-4901-4bbe-9421-084d7f5cff15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:43.322360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:43.323246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:43.324041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 57/112] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:43.341600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1f9d-4901-4bbe-9421-084d7f5cff15 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:43.407594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:43.414119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747050b-68c5-499f-a099-33002bf66e80 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:43.452304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e654d-ef96-4893-8cdc-a84269efb1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:43.453437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e654d-ef96-4893-8cdc-a84269efb1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.453914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e654d-ef96-4893-8cdc-a84269efb1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.454376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e654d-ef96-4893-8cdc-a84269efb1f9 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.455718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e654d-ef96-4893-8cdc-a84269efb1f9 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:43.457393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e654d-ef96-4893-8cdc-a84269efb1f9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:43.460309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 56/113] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:43 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:43.466651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e654d-ef96-4893-8cdc-a84269efb1f9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:43.500775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1f9d-4901-4bbe-9421-084d7f5cff15 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:43.505117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:43.552599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:43.553379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.553673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.553953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.554640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:43.554953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:43.556440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:43.566592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 58/114] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:43.567866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:43.583471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e654d-ef96-4893-8cdc-a84269efb1f9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:43.624914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef208bc-57e0-4b91-b871-fe42d64fb082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:43.625688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef208bc-57e0-4b91-b871-fe42d64fb082 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.626086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef208bc-57e0-4b91-b871-fe42d64fb082 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.626359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef208bc-57e0-4b91-b871-fe42d64fb082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.627267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef208bc-57e0-4b91-b871-fe42d64fb082 None admin] RBAC: Adding query filter params (role.id=c78dd2dd41224a73aec3edfe5d103a84, scope.project.id=822c049cabcb4edb8210aa9424fbb06a, user.id=27352662136f47c8845c3766faf50315) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:43.627564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef208bc-57e0-4b91-b871-fe42d64fb082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:43.629708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef208bc-57e0-4b91-b871-fe42d64fb082 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:43.644224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 57/115] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:43 2022] GET /identity/v3/role_assignments?user.id=27352662136f47c8845c3766faf50315&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=c78dd2dd41224a73aec3edfe5d103a84 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:43.644224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788928-3169-4a8f-9cde-698e7997df02 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.644224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788928-3169-4a8f-9cde-698e7997df02 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.644224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788928-3169-4a8f-9cde-698e7997df02 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.644224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 58/116] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:43.644224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78788928-3169-4a8f-9cde-698e7997df02 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:43.679078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704d774-90a5-469c-970a-62dacf605cb7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:43.686952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:43.716973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce18015-f73f-4ba2-a8d1-9547b459a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:43.717534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18015-f73f-4ba2-a8d1-9547b459a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.717815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18015-f73f-4ba2-a8d1-9547b459a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.718097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18015-f73f-4ba2-a8d1-9547b459a6a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.731454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce18015-f73f-4ba2-a8d1-9547b459a6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5ce18015-f73f-4ba2-a8d1-9547b459a6a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:43.736942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:43.738059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:43.739006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 59/117] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:17:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:43.740251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79a797-368a-4dc9-a279-f27ce461341e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.740251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79a797-368a-4dc9-a279-f27ce461341e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.740251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79a797-368a-4dc9-a279-f27ce461341e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.741559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 60/118] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:43.742441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb384d-d50d-4ca6-bb8e-e16be484854b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.742731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb384d-d50d-4ca6-bb8e-e16be484854b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.743754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb384d-d50d-4ca6-bb8e-e16be484854b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.767161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e3fb384d-d50d-4ca6-bb8e-e16be484854b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:43.768434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e3fb384d-d50d-4ca6-bb8e-e16be484854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:43.781666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78788928-3169-4a8f-9cde-698e7997df02 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:43.808006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:43.825475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805cab7-da29-441d-8db8-496d610f7163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:43.826126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805cab7-da29-441d-8db8-496d610f7163 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.826412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805cab7-da29-441d-8db8-496d610f7163 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.826744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805cab7-da29-441d-8db8-496d610f7163 None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.827842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805cab7-da29-441d-8db8-496d610f7163 None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:43.830035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805cab7-da29-441d-8db8-496d610f7163 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:43.831116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 59/119] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:43 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:43.831728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805cab7-da29-441d-8db8-496d610f7163 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:43.860072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 61/120] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:43.862010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a18f-dcca-4c51-a80d-824990134931 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:43.862372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a18f-dcca-4c51-a80d-824990134931 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:43.862654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a18f-dcca-4c51-a80d-824990134931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:43.889266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7116a18f-dcca-4c51-a80d-824990134931 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:43.890769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7116a18f-dcca-4c51-a80d-824990134931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:43.964004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805cab7-da29-441d-8db8-496d610f7163 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:43.964721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 62/121] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:43.965651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116a18f-dcca-4c51-a80d-824990134931 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:43.993678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:44.024464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:44.025090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:44.025375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:44.025650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:44.026335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:44.026641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:44.028039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:44.049865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 60/122] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:44.061250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:44.073072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116a18f-dcca-4c51-a80d-824990134931 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:44.112877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:44.134013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa0710-3fb8-47f7-abb5-853bcf584760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:44.134771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa0710-3fb8-47f7-abb5-853bcf584760 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:44.135113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa0710-3fb8-47f7-abb5-853bcf584760 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:44.135393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa0710-3fb8-47f7-abb5-853bcf584760 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:44.136590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa0710-3fb8-47f7-abb5-853bcf584760 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:44.138179 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa0710-3fb8-47f7-abb5-853bcf584760 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:44.140131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 63/123] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:43 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:44.150079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa0710-3fb8-47f7-abb5-853bcf584760 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:44.240037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b433c0-5609-4dd5-b62c-fe33f0c3d47d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:44.278466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c05a5-f375-481a-984d-6b2759a9a3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:44.279109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c05a5-f375-481a-984d-6b2759a9a3bc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:44.279389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c05a5-f375-481a-984d-6b2759a9a3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:44.279671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c05a5-f375-481a-984d-6b2759a9a3bc None admin] PATH_INFO: `/v3/users/23e13f53885b4bcd974e51d34ba1099e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:44.289684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c05a5-f375-481a-984d-6b2759a9a3bc None admin] RBAC: Authorizing `identity:get_user(user_id=23e13f53885b4bcd974e51d34ba1099e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:44.291182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c05a5-f375-481a-984d-6b2759a9a3bc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:44.293082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 61/124] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:44 2022] GET /identity/v3/users/23e13f53885b4bcd974e51d34ba1099e => generated 247 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:44.293675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c05a5-f375-481a-984d-6b2759a9a3bc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:44.356602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:44.363773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa0710-3fb8-47f7-abb5-853bcf584760 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:44.440108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586636f-8ed0-4f4c-a096-5d2734eb7e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:44.440815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586636f-8ed0-4f4c-a096-5d2734eb7e70 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:44.441102 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586636f-8ed0-4f4c-a096-5d2734eb7e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:44.441379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586636f-8ed0-4f4c-a096-5d2734eb7e70 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a/users/07197b17f7014c6192df2c1a77382f9b/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:44.443960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586636f-8ed0-4f4c-a096-5d2734eb7e70 None admin] RBAC: Authorizing `identity:create_grant(project_id=822c049cabcb4edb8210aa9424fbb06a, user_id=07197b17f7014c6192df2c1a77382f9b, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:44.446267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586636f-8ed0-4f4c-a096-5d2734eb7e70 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:44.459338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:44.465374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c05a5-f375-481a-984d-6b2759a9a3bc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:44.499917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea6a26-03e1-4675-8d97-46a8e30010d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:44.500586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6a26-03e1-4675-8d97-46a8e30010d9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:44.500887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6a26-03e1-4675-8d97-46a8e30010d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:44.501150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea6a26-03e1-4675-8d97-46a8e30010d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:44.511989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea6a26-03e1-4675-8d97-46a8e30010d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-35ea6a26-03e1-4675-8d97-46a8e30010d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:44.516854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:44.517736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:44.518490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 62/125] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:17:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:44.519159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea6a26-03e1-4675-8d97-46a8e30010d9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:44.545282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 64/126] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:17:44 2022] PUT /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a/users/07197b17f7014c6192df2c1a77382f9b/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:17:44.546391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586636f-8ed0-4f4c-a096-5d2734eb7e70 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:44.721504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea6a26-03e1-4675-8d97-46a8e30010d9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:44.789378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586636f-8ed0-4f4c-a096-5d2734eb7e70 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:44.792058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df3690-4a10-4a3f-a757-1d0ab1e9131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:44.792907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df3690-4a10-4a3f-a757-1d0ab1e9131b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:44.793340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df3690-4a10-4a3f-a757-1d0ab1e9131b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:44.793758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df3690-4a10-4a3f-a757-1d0ab1e9131b None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:44.795114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df3690-4a10-4a3f-a757-1d0ab1e9131b None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:44.796830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df3690-4a10-4a3f-a757-1d0ab1e9131b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:44.798930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 63/127] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:44 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:44.799851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df3690-4a10-4a3f-a757-1d0ab1e9131b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:44.843446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:44.844345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3803104-a4a7-4ea4-b25f-945189830d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:44.845456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3803104-a4a7-4ea4-b25f-945189830d3e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:44.845988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3803104-a4a7-4ea4-b25f-945189830d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:44.846496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3803104-a4a7-4ea4-b25f-945189830d3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:44.881448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3803104-a4a7-4ea4-b25f-945189830d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a3803104-a4a7-4ea4-b25f-945189830d3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:44.886504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:44.887587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:44.888518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:44.896202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 65/128] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:44.897603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3803104-a4a7-4ea4-b25f-945189830d3e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:44.931364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:44.941925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df3690-4a10-4a3f-a757-1d0ab1e9131b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:44.995417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:44.996091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:44.996405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:44.996687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:44.997379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:44.997688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:44.999097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:45.030285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 64/129] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:17:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:45.030971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:45.077056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3803104-a4a7-4ea4-b25f-945189830d3e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:45.123780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f28fa-afc8-45b7-8bb3-91bfd332e1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:45.124497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f28fa-afc8-45b7-8bb3-91bfd332e1cb None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.124785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f28fa-afc8-45b7-8bb3-91bfd332e1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.125061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f28fa-afc8-45b7-8bb3-91bfd332e1cb None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a/users/23e13f53885b4bcd974e51d34ba1099e/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.134341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f28fa-afc8-45b7-8bb3-91bfd332e1cb None admin] RBAC: Authorizing `identity:create_grant(project_id=822c049cabcb4edb8210aa9424fbb06a, user_id=23e13f53885b4bcd974e51d34ba1099e, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:45.135828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f28fa-afc8-45b7-8bb3-91bfd332e1cb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:45.196168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:45.213929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe7c20-055c-4bd1-afa6-a4af6bd1c647 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:45.243768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 66/130] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:17:44 2022] PUT /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a/users/23e13f53885b4bcd974e51d34ba1099e/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:17:45.243768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f28fa-afc8-45b7-8bb3-91bfd332e1cb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:45.287377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5042322-83e3-4e00-bc67-1f40a1cd3382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:45.290520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5042322-83e3-4e00-bc67-1f40a1cd3382 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.292118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5042322-83e3-4e00-bc67-1f40a1cd3382 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.292501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5042322-83e3-4e00-bc67-1f40a1cd3382 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.293899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5042322-83e3-4e00-bc67-1f40a1cd3382 None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:45.295595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5042322-83e3-4e00-bc67-1f40a1cd3382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:45.299143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5042322-83e3-4e00-bc67-1f40a1cd3382 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:45.319745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 65/131] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:45.321919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991b4ef-df7d-4f6e-a206-637590a14a89 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.322462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991b4ef-df7d-4f6e-a206-637590a14a89 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.325190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991b4ef-df7d-4f6e-a206-637590a14a89 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.327938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 66/132] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:45.332528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991b4ef-df7d-4f6e-a206-637590a14a89 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:45.431275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:45.465992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f28fa-afc8-45b7-8bb3-91bfd332e1cb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:45.506282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d62c36-11c9-400f-8b4c-5401c9425ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:45.507216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d62c36-11c9-400f-8b4c-5401c9425ebc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.507667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d62c36-11c9-400f-8b4c-5401c9425ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.508140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d62c36-11c9-400f-8b4c-5401c9425ebc None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.509652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d62c36-11c9-400f-8b4c-5401c9425ebc None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:45.511018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991b4ef-df7d-4f6e-a206-637590a14a89 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:45.511600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d62c36-11c9-400f-8b4c-5401c9425ebc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:45.513656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 67/133] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:45 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:45.516274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314c5e8-a72b-4be6-a5ac-c2f8d6b4a66c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.517038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314c5e8-a72b-4be6-a5ac-c2f8d6b4a66c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.517459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314c5e8-a72b-4be6-a5ac-c2f8d6b4a66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.539107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0314c5e8-a72b-4be6-a5ac-c2f8d6b4a66c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:45.540741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0314c5e8-a72b-4be6-a5ac-c2f8d6b4a66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:45.554704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b306a-b793-44fd-8c94-78f0243a7118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:45.555649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b306a-b793-44fd-8c94-78f0243a7118 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.555939 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b306a-b793-44fd-8c94-78f0243a7118 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.556370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b306a-b793-44fd-8c94-78f0243a7118 None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.557648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b306a-b793-44fd-8c94-78f0243a7118 None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:45.559328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b306a-b793-44fd-8c94-78f0243a7118 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:45.561316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 67/134] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:45 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:45.562049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b306a-b793-44fd-8c94-78f0243a7118 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:45.583331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:45.624753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 68/135] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:45.630413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314c5e8-a72b-4be6-a5ac-c2f8d6b4a66c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:45.704209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b306a-b793-44fd-8c94-78f0243a7118 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:45.776928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b50ac-22a7-4920-9837-b59368bae3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:45.777586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b50ac-22a7-4920-9837-b59368bae3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.777874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b50ac-22a7-4920-9837-b59368bae3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.778154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b50ac-22a7-4920-9837-b59368bae3d3 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.781225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b50ac-22a7-4920-9837-b59368bae3d3 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:45.782780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b50ac-22a7-4920-9837-b59368bae3d3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:45.784724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 68/136] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:45 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:45.790854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b50ac-22a7-4920-9837-b59368bae3d3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:45.793396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:45.855727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314c5e8-a72b-4be6-a5ac-c2f8d6b4a66c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:45.903590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6d764-5d5c-4ba5-b588-fb2d9de2fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:45.904614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6d764-5d5c-4ba5-b588-fb2d9de2fe03 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.905120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6d764-5d5c-4ba5-b588-fb2d9de2fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.905883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6d764-5d5c-4ba5-b588-fb2d9de2fe03 None admin] PATH_INFO: `/v3/projects/c648c3e2b08c45a588fe2c99a61f92f5` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.907742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6d764-5d5c-4ba5-b588-fb2d9de2fe03 None admin] RBAC: Authorizing `identity:get_project(project_id=c648c3e2b08c45a588fe2c99a61f92f5)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:45.909444 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6d764-5d5c-4ba5-b588-fb2d9de2fe03 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:45.912836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 69/137] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:45 2022] GET /identity/v3/projects/c648c3e2b08c45a588fe2c99a61f92f5 => generated 312 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:45.919902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57909-9f44-4bf7-8a50-2fe177d91556 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:45.920546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57909-9f44-4bf7-8a50-2fe177d91556 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:45.921024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57909-9f44-4bf7-8a50-2fe177d91556 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:45.922886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 70/138] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:45.923816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd57909-9f44-4bf7-8a50-2fe177d91556 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:45.996736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:46.030393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b50ac-22a7-4920-9837-b59368bae3d3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:46.077040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f769c9-2831-49a3-85b1-b6d1ac16ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:46.077720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f769c9-2831-49a3-85b1-b6d1ac16ba7e None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:46.078004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f769c9-2831-49a3-85b1-b6d1ac16ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:46.078279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f769c9-2831-49a3-85b1-b6d1ac16ba7e None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a/users/07197b17f7014c6192df2c1a77382f9b/roles/47878794de344232a226a95ce5771504` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:46.080626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f769c9-2831-49a3-85b1-b6d1ac16ba7e None admin] RBAC: Authorizing `identity:create_grant(project_id=822c049cabcb4edb8210aa9424fbb06a, user_id=07197b17f7014c6192df2c1a77382f9b, role_id=47878794de344232a226a95ce5771504)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:46.082430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f769c9-2831-49a3-85b1-b6d1ac16ba7e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:46.087333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd57909-9f44-4bf7-8a50-2fe177d91556 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:46.139104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aee4f-374c-4f7a-9b9e-c88d59ffbdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:46.139674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aee4f-374c-4f7a-9b9e-c88d59ffbdd7 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:46.139960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aee4f-374c-4f7a-9b9e-c88d59ffbdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:46.140271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aee4f-374c-4f7a-9b9e-c88d59ffbdd7 None admin] PATH_INFO: `/v3/projects/c648c3e2b08c45a588fe2c99a61f92f5/users/07197b17f7014c6192df2c1a77382f9b/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:46.142447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aee4f-374c-4f7a-9b9e-c88d59ffbdd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c648c3e2b08c45a588fe2c99a61f92f5, user_id=07197b17f7014c6192df2c1a77382f9b, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:46.143903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aee4f-374c-4f7a-9b9e-c88d59ffbdd7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:46.159218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 69/139] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:17:45 2022] PUT /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a/users/07197b17f7014c6192df2c1a77382f9b/roles/47878794de344232a226a95ce5771504 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:17:46.161566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a150712-a90b-4dc5-91b4-720c57798dc9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:46.162298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a150712-a90b-4dc5-91b4-720c57798dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:46.162784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a150712-a90b-4dc5-91b4-720c57798dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:46.187149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6a150712-a90b-4dc5-91b4-720c57798dc9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:46.189081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6a150712-a90b-4dc5-91b4-720c57798dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:46.219493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:46.229795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 71/140] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:17:45 2022] PUT /identity/v3/projects/c648c3e2b08c45a588fe2c99a61f92f5/users/07197b17f7014c6192df2c1a77382f9b/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:17:46.230758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aee4f-374c-4f7a-9b9e-c88d59ffbdd7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:46.284536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 70/141] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:46.414130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aee4f-374c-4f7a-9b9e-c88d59ffbdd7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:46.481823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a6c99-4190-489f-b1c5-f42487a5ebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:46.483895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a6c99-4190-489f-b1c5-f42487a5ebd9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:46.484606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a6c99-4190-489f-b1c5-f42487a5ebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:46.485270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a6c99-4190-489f-b1c5-f42487a5ebd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:46.487439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a6c99-4190-489f-b1c5-f42487a5ebd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:46.491620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a6c99-4190-489f-b1c5-f42487a5ebd9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:46.494819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 72/142] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:46.500998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a150712-a90b-4dc5-91b4-720c57798dc9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:46.643435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:46.671098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a150712-a90b-4dc5-91b4-720c57798dc9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:46.709579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a6c99-4190-489f-b1c5-f42487a5ebd9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:46.723834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e8288-b0fa-4f2c-a2e0-b80b9a082b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:46.724899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e8288-b0fa-4f2c-a2e0-b80b9a082b4e None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:46.725393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e8288-b0fa-4f2c-a2e0-b80b9a082b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:46.725871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e8288-b0fa-4f2c-a2e0-b80b9a082b4e None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:46.726825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e8288-b0fa-4f2c-a2e0-b80b9a082b4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:46.728518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e8288-b0fa-4f2c-a2e0-b80b9a082b4e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:46.808219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:46.843089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 71/143] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:17:46 2022] POST /identity/v3/users => generated 283 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:46.847383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a6c99-4190-489f-b1c5-f42487a5ebd9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:46.899454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c675126-7f11-46fb-ba6d-e6c46f35269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:46.900117 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675126-7f11-46fb-ba6d-e6c46f35269c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:46.900423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675126-7f11-46fb-ba6d-e6c46f35269c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:46.900707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c675126-7f11-46fb-ba6d-e6c46f35269c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:46.917013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c675126-7f11-46fb-ba6d-e6c46f35269c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2c675126-7f11-46fb-ba6d-e6c46f35269c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:46.923498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:46.924381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:17:46.925148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:46.927416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 73/144] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:46.930813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e8288-b0fa-4f2c-a2e0-b80b9a082b4e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:46.978918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86060d6-d9f7-47da-8572-fbb55fb6ee36 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:46.979271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86060d6-d9f7-47da-8572-fbb55fb6ee36 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:46.979482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86060d6-d9f7-47da-8572-fbb55fb6ee36 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:46.980953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 74/145] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:46.987467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13268f06-f441-4447-bed9-ea8c3edf6986 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:46.987789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13268f06-f441-4447-bed9-ea8c3edf6986 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:46.988021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13268f06-f441-4447-bed9-ea8c3edf6986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:47.026508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-13268f06-f441-4447-bed9-ea8c3edf6986 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:47.027956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-13268f06-f441-4447-bed9-ea8c3edf6986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:47.156284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e8288-b0fa-4f2c-a2e0-b80b9a082b4e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:47.182347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 75/146] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:47.205603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:47.213838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:47.214539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:47.214821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:47.215096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:47.215783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:47.216083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:47.217498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:47.229112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 72/147] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:47.235184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13268f06-f441-4447-bed9-ea8c3edf6986 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:47.333709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:47.384097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13268f06-f441-4447-bed9-ea8c3edf6986 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:47.426323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ee560-fd22-4ecf-9c6f-ce62790f6b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:47.426973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ee560-fd22-4ecf-9c6f-ce62790f6b6f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:47.427249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ee560-fd22-4ecf-9c6f-ce62790f6b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:47.427521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ee560-fd22-4ecf-9c6f-ce62790f6b6f None admin] PATH_INFO: `/v3/users/27352662136f47c8845c3766faf50315` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:47.443469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ee560-fd22-4ecf-9c6f-ce62790f6b6f None admin] RBAC: Authorizing `identity:get_user(user_id=27352662136f47c8845c3766faf50315)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:47.443469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ee560-fd22-4ecf-9c6f-ce62790f6b6f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:47.443469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 76/148] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:47 2022] GET /identity/v3/users/27352662136f47c8845c3766faf50315 => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:47.443861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:47.544560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ee560-fd22-4ecf-9c6f-ce62790f6b6f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:47.596132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a36b2-fd88-4f51-bea0-1d77f820c2d0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:47.652787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50b008-aa71-48d7-b4ac-c191838668f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:47.653489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b008-aa71-48d7-b4ac-c191838668f9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:47.653777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b008-aa71-48d7-b4ac-c191838668f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:47.654055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b008-aa71-48d7-b4ac-c191838668f9 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:47.655275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef50b008-aa71-48d7-b4ac-c191838668f9 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:47.656780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef50b008-aa71-48d7-b4ac-c191838668f9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:47.658467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 73/149] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:47 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:47.669678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50b008-aa71-48d7-b4ac-c191838668f9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:47.763789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ee560-fd22-4ecf-9c6f-ce62790f6b6f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:47.796450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:47.809729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31069eca-83fb-49c1-965d-d7398d06bf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:47.814178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31069eca-83fb-49c1-965d-d7398d06bf2c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:47.814178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31069eca-83fb-49c1-965d-d7398d06bf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:47.814178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31069eca-83fb-49c1-965d-d7398d06bf2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:47.821174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:47.833320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31069eca-83fb-49c1-965d-d7398d06bf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-31069eca-83fb-49c1-965d-d7398d06bf2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:47.847994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:47.849022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:47.849799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 77/150] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:17:47.856431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31069eca-83fb-49c1-965d-d7398d06bf2c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:47.952233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50b008-aa71-48d7-b4ac-c191838668f9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:47.985759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31069eca-83fb-49c1-965d-d7398d06bf2c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:48.027223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:48.027832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:48.028110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:48.028411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:48.029080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:48.029725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:48.031855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:48.041964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 78/151] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:48.042374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d1094-0d3e-472f-aab6-9c66dc103b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:48.043120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d1094-0d3e-472f-aab6-9c66dc103b97 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:48.043410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d1094-0d3e-472f-aab6-9c66dc103b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:48.043687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d1094-0d3e-472f-aab6-9c66dc103b97 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a/users/27352662136f47c8845c3766faf50315/roles/c78dd2dd41224a73aec3edfe5d103a84` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:48.048076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:48.052028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d1094-0d3e-472f-aab6-9c66dc103b97 None admin] RBAC: Authorizing `identity:create_grant(project_id=822c049cabcb4edb8210aa9424fbb06a, user_id=27352662136f47c8845c3766faf50315, role_id=c78dd2dd41224a73aec3edfe5d103a84)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:48.052028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d1094-0d3e-472f-aab6-9c66dc103b97 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:48.103850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:48.150969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 74/152] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:17:47 2022] PUT /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a/users/27352662136f47c8845c3766faf50315/roles/c78dd2dd41224a73aec3edfe5d103a84 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:17:48.161797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ce2d6-c0cb-4598-86aa-e6bdcdb62d20 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:48.205530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081719-4c2b-44c8-ac3c-b0852847ac83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:48.206172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081719-4c2b-44c8-ac3c-b0852847ac83 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:48.206451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081719-4c2b-44c8-ac3c-b0852847ac83 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:48.206720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081719-4c2b-44c8-ac3c-b0852847ac83 None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:48.208537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb081719-4c2b-44c8-ac3c-b0852847ac83 None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:48.210151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb081719-4c2b-44c8-ac3c-b0852847ac83 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:48.211942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 79/153] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:48 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:48.217920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d1094-0d3e-472f-aab6-9c66dc103b97 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:48.349078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d1094-0d3e-472f-aab6-9c66dc103b97 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:48.372700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:48.392060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df326b3-43c6-435c-8e29-a7bfc5ebd030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:48.392773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df326b3-43c6-435c-8e29-a7bfc5ebd030 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:48.393054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df326b3-43c6-435c-8e29-a7bfc5ebd030 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:48.393330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df326b3-43c6-435c-8e29-a7bfc5ebd030 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:48.430239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb66c8-be55-4599-8eb6-8257e8ce4870 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:48.430564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb66c8-be55-4599-8eb6-8257e8ce4870 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:48.430785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb66c8-be55-4599-8eb6-8257e8ce4870 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:48.432426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 80/154] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:48.438206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd0b7e-75a1-4677-b09d-038c8eefe8a3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:48.438494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd0b7e-75a1-4677-b09d-038c8eefe8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:48.438746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd0b7e-75a1-4677-b09d-038c8eefe8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:48.477729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b9bd0b7e-75a1-4677-b09d-038c8eefe8a3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:48.479255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b9bd0b7e-75a1-4677-b09d-038c8eefe8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:48.526623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df326b3-43c6-435c-8e29-a7bfc5ebd030 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:48.528209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df326b3-43c6-435c-8e29-a7bfc5ebd030 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:48.598949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 81/155] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:48.685839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 75/156] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:48.691571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd0b7e-75a1-4677-b09d-038c8eefe8a3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:48.767108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:48.838406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd0b7e-75a1-4677-b09d-038c8eefe8a3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:48.883429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a1630-8a28-4e62-8e30-26cb2addd294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:48.884082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1630-8a28-4e62-8e30-26cb2addd294 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:48.884383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1630-8a28-4e62-8e30-26cb2addd294 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:48.884657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1630-8a28-4e62-8e30-26cb2addd294 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:48.885819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a1630-8a28-4e62-8e30-26cb2addd294 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:48.887280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a1630-8a28-4e62-8e30-26cb2addd294 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:48.888985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 82/157] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:48 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:48.895209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df326b3-43c6-435c-8e29-a7bfc5ebd030 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:48.927765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:49.023715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df326b3-43c6-435c-8e29-a7bfc5ebd030 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:49.063865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:49.064510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:49.064793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:49.065064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:49.065804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] RBAC: Adding query filter params (role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=822c049cabcb4edb8210aa9424fbb06a, user.id=07197b17f7014c6192df2c1a77382f9b) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:49.066193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:49.068429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:49.075413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 76/158] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:48 2022] GET /identity/v3/role_assignments?user.id=07197b17f7014c6192df2c1a77382f9b&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:49.137732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a1630-8a28-4e62-8e30-26cb2addd294 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:49.273502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a1630-8a28-4e62-8e30-26cb2addd294 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:49.298324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:49.320484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466634-f5a3-450a-a29d-06aabc87224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:49.321132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466634-f5a3-450a-a29d-06aabc87224a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:49.321415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466634-f5a3-450a-a29d-06aabc87224a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:49.321683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38466634-f5a3-450a-a29d-06aabc87224a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:49.338686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38466634-f5a3-450a-a29d-06aabc87224a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-38466634-f5a3-450a-a29d-06aabc87224a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:49.349127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:49.350063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:49.350906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 83/159] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:49.358220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:49.457467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:49.481161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6c41-78b6-423d-8c75-cb2e75179df8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:49.523843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e8cd0-3705-40e5-ab5f-0811fbcc2cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:49.524505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e8cd0-3705-40e5-ab5f-0811fbcc2cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:49.524786 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e8cd0-3705-40e5-ab5f-0811fbcc2cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:49.525061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e8cd0-3705-40e5-ab5f-0811fbcc2cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:49.525786 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e8cd0-3705-40e5-ab5f-0811fbcc2cb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:49.526085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e8cd0-3705-40e5-ab5f-0811fbcc2cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:49.527523 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e8cd0-3705-40e5-ab5f-0811fbcc2cb3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:49.537349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 77/160] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:49.542785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466634-f5a3-450a-a29d-06aabc87224a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:49.564352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653048-957c-49ff-8206-6b306a148ef5 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:49.564749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653048-957c-49ff-8206-6b306a148ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:49.565220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653048-957c-49ff-8206-6b306a148ef5 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:49.567564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 78/161] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:49.576320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42d730-6e31-4d39-b7a5-4624f8fb7fb0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:49.576619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42d730-6e31-4d39-b7a5-4624f8fb7fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:49.576869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42d730-6e31-4d39-b7a5-4624f8fb7fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:49.616582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-da42d730-6e31-4d39-b7a5-4624f8fb7fb0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:49.617805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-da42d730-6e31-4d39-b7a5-4624f8fb7fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:49.780577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 79/162] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:49.781334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38466634-f5a3-450a-a29d-06aabc87224a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:49.873099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58692813-80dc-4563-8596-9f3edf3a2e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:49.873707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58692813-80dc-4563-8596-9f3edf3a2e5b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:49.873994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58692813-80dc-4563-8596-9f3edf3a2e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:49.874273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58692813-80dc-4563-8596-9f3edf3a2e5b None admin] PATH_INFO: `/v3/users/23e13f53885b4bcd974e51d34ba1099e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:49.875448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58692813-80dc-4563-8596-9f3edf3a2e5b None admin] RBAC: Authorizing `identity:get_user(user_id=23e13f53885b4bcd974e51d34ba1099e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:49.876948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58692813-80dc-4563-8596-9f3edf3a2e5b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:49.878559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 84/163] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:49 2022] GET /identity/v3/users/23e13f53885b4bcd974e51d34ba1099e => generated 247 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:49.884795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42d730-6e31-4d39-b7a5-4624f8fb7fb0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:49.925743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac46202-b005-4cfd-a363-41e5ec38dce7 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:49.926052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac46202-b005-4cfd-a363-41e5ec38dce7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:49.927027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac46202-b005-4cfd-a363-41e5ec38dce7 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:49.929008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 85/164] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:49.965983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12060024-e8d8-4333-ad69-a8507813e70a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:49.966317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12060024-e8d8-4333-ad69-a8507813e70a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:49.966550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12060024-e8d8-4333-ad69-a8507813e70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:49.999508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-12060024-e8d8-4333-ad69-a8507813e70a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:50.000833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-12060024-e8d8-4333-ad69-a8507813e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:50.039427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:50.083510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:50.114421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42d730-6e31-4d39-b7a5-4624f8fb7fb0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:50.138201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 86/165] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:50.235641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82124c9f-5c72-4d73-aac8-8710fec98ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:50.236681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82124c9f-5c72-4d73-aac8-8710fec98ddb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:50.237190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82124c9f-5c72-4d73-aac8-8710fec98ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:50.237674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82124c9f-5c72-4d73-aac8-8710fec98ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:50.376223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82124c9f-5c72-4d73-aac8-8710fec98ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:50.377750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82124c9f-5c72-4d73-aac8-8710fec98ddb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:50.489604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12060024-e8d8-4333-ad69-a8507813e70a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:50.516533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:50.555794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 80/166] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:50.561677 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82124c9f-5c72-4d73-aac8-8710fec98ddb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:50.722069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82124c9f-5c72-4d73-aac8-8710fec98ddb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:50.733527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12060024-e8d8-4333-ad69-a8507813e70a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:50.789077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23fca9-39cd-4410-81f2-1c67d22058b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:50.789751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23fca9-39cd-4410-81f2-1c67d22058b1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:50.790014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23fca9-39cd-4410-81f2-1c67d22058b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:50.790289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23fca9-39cd-4410-81f2-1c67d22058b1 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:50.791623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23fca9-39cd-4410-81f2-1c67d22058b1 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:50.793230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23fca9-39cd-4410-81f2-1c67d22058b1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:50.795344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 81/167] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:50 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:50.801819 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23fca9-39cd-4410-81f2-1c67d22058b1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:50.814952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3753ed-af26-42d8-8c9d-5ae59e804355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:50.816532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3753ed-af26-42d8-8c9d-5ae59e804355 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:50.817271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3753ed-af26-42d8-8c9d-5ae59e804355 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:50.817911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3753ed-af26-42d8-8c9d-5ae59e804355 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:50.853390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3753ed-af26-42d8-8c9d-5ae59e804355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-cc3753ed-af26-42d8-8c9d-5ae59e804355 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:50.858999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:50.859926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:50.865327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:50.867579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 87/168] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:17:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:50.868277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:50.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3753ed-af26-42d8-8c9d-5ae59e804355 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:50.935846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:50.968220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23fca9-39cd-4410-81f2-1c67d22058b1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:51.028369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3753ed-af26-42d8-8c9d-5ae59e804355 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:51.034536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:51.035367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:51.036079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:51.036670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:51.037572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=822c049cabcb4edb8210aa9424fbb06a, user.id=23e13f53885b4bcd974e51d34ba1099e) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:51.038032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:51.048208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:51.058844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 82/169] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:50 2022] GET /identity/v3/role_assignments?user.id=23e13f53885b4bcd974e51d34ba1099e&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:51.060150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:51.078448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:51.079128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:51.079412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:51.079686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:51.080419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:51.080722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:51.084677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:51.108834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 88/170] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:17:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:51.115887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:51.261596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bccf44-b836-4a4e-8327-0e5f760a6977 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:51.272625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:51.320827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496cd4f-a0a2-4571-bd5f-ed23c780575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:51.321444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496cd4f-a0a2-4571-bd5f-ed23c780575f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:51.321728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496cd4f-a0a2-4571-bd5f-ed23c780575f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:51.322006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496cd4f-a0a2-4571-bd5f-ed23c780575f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:51.339422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496cd4f-a0a2-4571-bd5f-ed23c780575f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b496cd4f-a0a2-4571-bd5f-ed23c780575f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:51.345385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:51.346297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:51.347087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 83/171] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:51.350460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:51.355205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496cd4f-a0a2-4571-bd5f-ed23c780575f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:51.374530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ddd49-dbad-424a-8d40-d5ad6cea3def None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:51.462264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79659acd-a42d-48a6-8306-e1d903b75b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:51.462264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659acd-a42d-48a6-8306-e1d903b75b78 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:51.462264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659acd-a42d-48a6-8306-e1d903b75b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:51.462264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659acd-a42d-48a6-8306-e1d903b75b78 None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:51.463638 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79659acd-a42d-48a6-8306-e1d903b75b78 None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:51.465207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79659acd-a42d-48a6-8306-e1d903b75b78 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:51.468093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 89/172] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:51 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:51.477413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79659acd-a42d-48a6-8306-e1d903b75b78 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:51.560631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496cd4f-a0a2-4571-bd5f-ed23c780575f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:51.628161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:51.628726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:51.629005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:51.629426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:51.630145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:51.630470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:51.632866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:51.652596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 84/173] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:51.659150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:51.703777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:51.766835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79659acd-a42d-48a6-8306-e1d903b75b78 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:51.842090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12973c5d-d33e-4926-a5cd-08261ebb3b0e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:51.846681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061543fe-5cd8-458a-b4ad-b4ed81524760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:51.847872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061543fe-5cd8-458a-b4ad-b4ed81524760 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:51.849491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061543fe-5cd8-458a-b4ad-b4ed81524760 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:51.850754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061543fe-5cd8-458a-b4ad-b4ed81524760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:51.915331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e82bff-b6b0-427b-b76f-329f1ca7e616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:51.916370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82bff-b6b0-427b-b76f-329f1ca7e616 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:51.916860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82bff-b6b0-427b-b76f-329f1ca7e616 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:51.917642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e82bff-b6b0-427b-b76f-329f1ca7e616 None admin] PATH_INFO: `/v3/users/07197b17f7014c6192df2c1a77382f9b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:51.919270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e82bff-b6b0-427b-b76f-329f1ca7e616 None admin] RBAC: Authorizing `identity:get_user(user_id=07197b17f7014c6192df2c1a77382f9b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:51.921008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e82bff-b6b0-427b-b76f-329f1ca7e616 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:51.923429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 85/174] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:51 2022] GET /identity/v3/users/07197b17f7014c6192df2c1a77382f9b => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:51.929710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16dbc0-3e3e-4b77-8f98-b6461a9d2079 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:51.930284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16dbc0-3e3e-4b77-8f98-b6461a9d2079 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:51.930762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16dbc0-3e3e-4b77-8f98-b6461a9d2079 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:51.932362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 86/175] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:51.935223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16dbc0-3e3e-4b77-8f98-b6461a9d2079 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:51.938481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:52.039527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061543fe-5cd8-458a-b4ad-b4ed81524760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:52.041086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061543fe-5cd8-458a-b4ad-b4ed81524760 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:52.060456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:52.067712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16dbc0-3e3e-4b77-8f98-b6461a9d2079 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:52.135359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a66abf-9deb-459c-93ef-aed2bbc8fc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:52.136220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a66abf-9deb-459c-93ef-aed2bbc8fc56 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.136638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a66abf-9deb-459c-93ef-aed2bbc8fc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.137245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a66abf-9deb-459c-93ef-aed2bbc8fc56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.147754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 90/176] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:17:52.149303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbfee1-4767-49b6-a044-f8f7e7a09495 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.149588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbfee1-4767-49b6-a044-f8f7e7a09495 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.149866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cbfee1-4767-49b6-a044-f8f7e7a09495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.168500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-63cbfee1-4767-49b6-a044-f8f7e7a09495 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:52.169611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-63cbfee1-4767-49b6-a044-f8f7e7a09495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:52.224882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a66abf-9deb-459c-93ef-aed2bbc8fc56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:52.226775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a66abf-9deb-459c-93ef-aed2bbc8fc56 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:52.230148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:52.241006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 91/177] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:52.249824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b502a72-18cc-4185-9245-4b6cf6281a3c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.250115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b502a72-18cc-4185-9245-4b6cf6281a3c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.250392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b502a72-18cc-4185-9245-4b6cf6281a3c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.251787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 92/178] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:52.252842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1f341-75d2-4c40-95cd-1ff2dc71f138 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.253125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1f341-75d2-4c40-95cd-1ff2dc71f138 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.253396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1f341-75d2-4c40-95cd-1ff2dc71f138 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.254873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 93/179] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:52.255672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1f341-75d2-4c40-95cd-1ff2dc71f138 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:52.362697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:52.385397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 87/180] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:52.385397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3175d16-890d-471a-a9e1-b467de6159ee None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.385397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3175d16-890d-471a-a9e1-b467de6159ee None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.385397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3175d16-890d-471a-a9e1-b467de6159ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.403369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f3175d16-890d-471a-a9e1-b467de6159ee None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:52.404613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f3175d16-890d-471a-a9e1-b467de6159ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:52.411881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1f341-75d2-4c40-95cd-1ff2dc71f138 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:52.451083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e4273-748a-4852-9d37-c7230c0c5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:52.452064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4273-748a-4852-9d37-c7230c0c5adf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.452578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4273-748a-4852-9d37-c7230c0c5adf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.453053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4273-748a-4852-9d37-c7230c0c5adf None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.454388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e4273-748a-4852-9d37-c7230c0c5adf None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:52.456125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e4273-748a-4852-9d37-c7230c0c5adf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:52.459498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 94/181] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:52 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:52.460906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072dee2-daeb-4c70-aece-d30e056be062 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.461262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072dee2-daeb-4c70-aece-d30e056be062 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.461608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072dee2-daeb-4c70-aece-d30e056be062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.481180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e072dee2-daeb-4c70-aece-d30e056be062 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:52.482401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e072dee2-daeb-4c70-aece-d30e056be062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:52.484472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 88/182] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:52.491854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77f46-9bf9-4d57-b3fa-8da55cd3a727 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.491854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77f46-9bf9-4d57-b3fa-8da55cd3a727 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.491854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f77f46-9bf9-4d57-b3fa-8da55cd3a727 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.491854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 89/183] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:52.491854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f77f46-9bf9-4d57-b3fa-8da55cd3a727 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:52.561773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:52.574973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 95/184] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:52.584766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072dee2-daeb-4c70-aece-d30e056be062 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:52.626826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f77f46-9bf9-4d57-b3fa-8da55cd3a727 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:52.677808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:52.691147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cc4ba-ec3d-4412-aad0-68b6b5efb2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:52.693104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc4ba-ec3d-4412-aad0-68b6b5efb2bc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.693893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc4ba-ec3d-4412-aad0-68b6b5efb2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.694332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc4ba-ec3d-4412-aad0-68b6b5efb2bc None admin] PATH_INFO: `/v3/projects/c648c3e2b08c45a588fe2c99a61f92f5` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.696558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc4ba-ec3d-4412-aad0-68b6b5efb2bc None admin] RBAC: Authorizing `identity:get_project(project_id=c648c3e2b08c45a588fe2c99a61f92f5)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:52.699686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc4ba-ec3d-4412-aad0-68b6b5efb2bc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:52.710568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 90/185] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:52 2022] GET /identity/v3/projects/c648c3e2b08c45a588fe2c99a61f92f5 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:52.710568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f5d3-3b70-433c-aba3-b21c7f0aae15 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.710568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f5d3-3b70-433c-aba3-b21c7f0aae15 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.710568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1f5d3-3b70-433c-aba3-b21c7f0aae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.731734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072dee2-daeb-4c70-aece-d30e056be062 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:52.739586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9be1f5d3-3b70-433c-aba3-b21c7f0aae15 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:52.742001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9be1f5d3-3b70-433c-aba3-b21c7f0aae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:52.789757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:52.790773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:52.791854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:52.792372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:52.793344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] RBAC: Adding query filter params (role.id=47878794de344232a226a95ce5771504, scope.project.id=822c049cabcb4edb8210aa9424fbb06a, user.id=07197b17f7014c6192df2c1a77382f9b) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:52.794361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:52.796753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:52.805887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 96/186] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:52 2022] GET /identity/v3/role_assignments?user.id=07197b17f7014c6192df2c1a77382f9b&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=47878794de344232a226a95ce5771504 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:52.807606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:52.832655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 91/187] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:52.833778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f5d3-3b70-433c-aba3-b21c7f0aae15 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:52.903381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:52.956879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a9900-0a16-468e-a6d9-5e409ccff7c5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:52.989007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1f5d3-3b70-433c-aba3-b21c7f0aae15 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.005080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f05ac0-be28-4532-b269-5992571ddab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.006002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f05ac0-be28-4532-b269-5992571ddab2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.006316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f05ac0-be28-4532-b269-5992571ddab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.006578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f05ac0-be28-4532-b269-5992571ddab2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.025258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f05ac0-be28-4532-b269-5992571ddab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-f3f05ac0-be28-4532-b269-5992571ddab2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:53.030564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:53.031461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:53.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 97/188] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:17:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:53.032986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f05ac0-be28-4532-b269-5992571ddab2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.039074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:53.042653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.043277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.043556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.043839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.044715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] RBAC: Adding query filter params (role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=c648c3e2b08c45a588fe2c99a61f92f5, user.id=07197b17f7014c6192df2c1a77382f9b) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:53.045025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:53.046548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:53.055602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 92/189] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:52 2022] GET /identity/v3/role_assignments?user.id=07197b17f7014c6192df2c1a77382f9b&scope.project.id=c648c3e2b08c45a588fe2c99a61f92f5&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:53.056496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.167870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f05ac0-be28-4532-b269-5992571ddab2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.185615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:53.193244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe9509-67f9-4a7f-9143-0d0f19b01512 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.219437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62ad47-f524-4c66-8a3e-3fc809fc602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.220219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62ad47-f524-4c66-8a3e-3fc809fc602a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.220513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62ad47-f524-4c66-8a3e-3fc809fc602a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.220788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62ad47-f524-4c66-8a3e-3fc809fc602a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.221949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62ad47-f524-4c66-8a3e-3fc809fc602a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:53.224451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62ad47-f524-4c66-8a3e-3fc809fc602a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:53.225209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 98/190] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:53.226069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62ad47-f524-4c66-8a3e-3fc809fc602a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.236911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f62c4-f63e-4ae6-988c-cb41fc31b51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.237560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f62c4-f63e-4ae6-988c-cb41fc31b51d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.237841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f62c4-f63e-4ae6-988c-cb41fc31b51d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.239367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f62c4-f63e-4ae6-988c-cb41fc31b51d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.251334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f62c4-f63e-4ae6-988c-cb41fc31b51d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-cf9f62c4-f63e-4ae6-988c-cb41fc31b51d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:53.258115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:53.259013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 93/191] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:53.260473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f62c4-f63e-4ae6-988c-cb41fc31b51d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.322461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:53.354729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62ad47-f524-4c66-8a3e-3fc809fc602a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.379319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f62c4-f63e-4ae6-988c-cb41fc31b51d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.395271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.396101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.396429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.396708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.397392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:53.397697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:53.399093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:53.407369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 99/192] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:53.408911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.415324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d1a17-be9d-4f0b-8c88-ee0c382f53d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.415964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d1a17-be9d-4f0b-8c88-ee0c382f53d5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.416275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d1a17-be9d-4f0b-8c88-ee0c382f53d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.416548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d1a17-be9d-4f0b-8c88-ee0c382f53d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.428171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d1a17-be9d-4f0b-8c88-ee0c382f53d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-520d1a17-be9d-4f0b-8c88-ee0c382f53d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:53.433475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:53.434414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:53.435199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 94/193] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:53.440367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d1a17-be9d-4f0b-8c88-ee0c382f53d5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.462207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:53.519624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3de2f4-db86-4822-8588-a3c69276289e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.544064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d1a17-be9d-4f0b-8c88-ee0c382f53d5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.554389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f84d5-5272-47fc-b7dd-8460fc4d1fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.554997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f84d5-5272-47fc-b7dd-8460fc4d1fca None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.555272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f84d5-5272-47fc-b7dd-8460fc4d1fca None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.555543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f84d5-5272-47fc-b7dd-8460fc4d1fca None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.555931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:53.556211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f84d5-5272-47fc-b7dd-8460fc4d1fca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:53.557676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f84d5-5272-47fc-b7dd-8460fc4d1fca None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:53.578835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.579445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.579734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.580016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.580737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:53.581044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:53.582505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:53.591811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 95/194] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:53.592730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.693381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 100/195] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:17:53 2022] POST /identity/v3/users => generated 297 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:53.694215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f84d5-5272-47fc-b7dd-8460fc4d1fca None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.694620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b01df3-bdce-4fb6-914a-626923163bb2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.735254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66464ceb-d456-4612-86a1-31b2fac56592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.736215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464ceb-d456-4612-86a1-31b2fac56592 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.736518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464ceb-d456-4612-86a1-31b2fac56592 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.736798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66464ceb-d456-4612-86a1-31b2fac56592 None admin] PATH_INFO: `/v3/users/8d8f4077ceea4670a530f88d68330e4e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.755421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66464ceb-d456-4612-86a1-31b2fac56592 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8f4077ceea4670a530f88d68330e4e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:53.757025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66464ceb-d456-4612-86a1-31b2fac56592 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:53.758915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 96/196] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/users/8d8f4077ceea4670a530f88d68330e4e => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:53.760021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66464ceb-d456-4612-86a1-31b2fac56592 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.796333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:53.841983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:53.847269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f84d5-5272-47fc-b7dd-8460fc4d1fca None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.887614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66464ceb-d456-4612-86a1-31b2fac56592 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:53.888089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.888919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.889320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.891555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.891555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:53.891555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:53.892107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:53.902484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 101/197] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:53.903486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:53.932958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808ca49-44ea-4b88-b55f-dee9d57f449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:53.933597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808ca49-44ea-4b88-b55f-dee9d57f449c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:53.933873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808ca49-44ea-4b88-b55f-dee9d57f449c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:53.934144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808ca49-44ea-4b88-b55f-dee9d57f449c None admin] PATH_INFO: `/v3/users/8d8f4077ceea4670a530f88d68330e4e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:53.935290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808ca49-44ea-4b88-b55f-dee9d57f449c None admin] RBAC: Authorizing `identity:get_user(user_id=8d8f4077ceea4670a530f88d68330e4e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:53.936771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808ca49-44ea-4b88-b55f-dee9d57f449c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:53.938447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 97/198] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/users/8d8f4077ceea4670a530f88d68330e4e => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:53.939284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808ca49-44ea-4b88-b55f-dee9d57f449c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:54.041187 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7edfb-d3a9-4e7b-b74f-79b9b5f47c01 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:54.072641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:54.078216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808ca49-44ea-4b88-b55f-dee9d57f449c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:54.094690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8a27-ac6f-4c7e-ac5e-4a010265ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:54.095505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8a27-ac6f-4c7e-ac5e-4a010265ef37 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:54.096002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8a27-ac6f-4c7e-ac5e-4a010265ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:54.096330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8a27-ac6f-4c7e-ac5e-4a010265ef37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:54.121644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4062d90-bfb8-43eb-be45-1144dd54495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:54.122340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4062d90-bfb8-43eb-be45-1144dd54495f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:54.122619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4062d90-bfb8-43eb-be45-1144dd54495f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:54.122899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4062d90-bfb8-43eb-be45-1144dd54495f None admin] PATH_INFO: `/v3/users/27352662136f47c8845c3766faf50315` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:54.124373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4062d90-bfb8-43eb-be45-1144dd54495f None admin] RBAC: Authorizing `identity:get_user(user_id=27352662136f47c8845c3766faf50315)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:54.125838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4062d90-bfb8-43eb-be45-1144dd54495f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:54.127748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 98/199] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/users/27352662136f47c8845c3766faf50315 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:54.134255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4062d90-bfb8-43eb-be45-1144dd54495f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:54.141793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:54.205638 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8a27-ac6f-4c7e-ac5e-4a010265ef37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:54.207847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8a27-ac6f-4c7e-ac5e-4a010265ef37 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:54.289425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4062d90-bfb8-43eb-be45-1144dd54495f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:54.328570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ff680-611c-4b73-ad74-ea113df9490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:54.329532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ff680-611c-4b73-ad74-ea113df9490a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:54.330018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ff680-611c-4b73-ad74-ea113df9490a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:54.330500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ff680-611c-4b73-ad74-ea113df9490a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:54.345550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 102/200] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:54.345550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8a27-ac6f-4c7e-ac5e-4a010265ef37 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:54.375213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:54.414605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:54.421529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ff680-611c-4b73-ad74-ea113df9490a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:54.423299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ff680-611c-4b73-ad74-ea113df9490a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:54.468705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8a27-ac6f-4c7e-ac5e-4a010265ef37 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:54.517059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b379907-0b5d-49f4-88f6-74f0847bae26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:54.518089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b379907-0b5d-49f4-88f6-74f0847bae26 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:54.518669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b379907-0b5d-49f4-88f6-74f0847bae26 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:54.519141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b379907-0b5d-49f4-88f6-74f0847bae26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:54.560001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 99/201] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:54.564883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ff680-611c-4b73-ad74-ea113df9490a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:54.618387 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b379907-0b5d-49f4-88f6-74f0847bae26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:54.620400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b379907-0b5d-49f4-88f6-74f0847bae26 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:54.648916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:54.690338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ff680-611c-4b73-ad74-ea113df9490a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:54.714246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:54.732775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376061-f1fd-4425-9cf2-a404f0ec2034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:54.733792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376061-f1fd-4425-9cf2-a404f0ec2034 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:54.734303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376061-f1fd-4425-9cf2-a404f0ec2034 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:54.734787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85376061-f1fd-4425-9cf2-a404f0ec2034 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:54.736561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85376061-f1fd-4425-9cf2-a404f0ec2034 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:54.738256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85376061-f1fd-4425-9cf2-a404f0ec2034 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:54.740942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 100/202] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:54 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:54.741954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376061-f1fd-4425-9cf2-a404f0ec2034 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:54.777468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 103/203] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:17:54.778481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b379907-0b5d-49f4-88f6-74f0847bae26 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:54.876287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85376061-f1fd-4425-9cf2-a404f0ec2034 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:54.917261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760089b-3a32-4117-a3ec-f92251087647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:54.918112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760089b-3a32-4117-a3ec-f92251087647 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:54.918746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760089b-3a32-4117-a3ec-f92251087647 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:54.919103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760089b-3a32-4117-a3ec-f92251087647 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:54.921071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760089b-3a32-4117-a3ec-f92251087647 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:54.923618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760089b-3a32-4117-a3ec-f92251087647 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:54.925855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 101/204] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:54 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:54.926807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b379907-0b5d-49f4-88f6-74f0847bae26 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:54.932568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760089b-3a32-4117-a3ec-f92251087647 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:54.936483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:54.990204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:54.991029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:54.991407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:54.991779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:54.993500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=7f770973effe4795978f1b5d9c097b92, user.id=8d8f4077ceea4670a530f88d68330e4e) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:54.993944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:54.996195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:55.005391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 104/205] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:54 2022] GET /identity/v3/role_assignments?user.id=8d8f4077ceea4670a530f88d68330e4e&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:55.006560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:55.048386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:55.048633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760089b-3a32-4117-a3ec-f92251087647 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:55.095304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6179d-9b61-455a-a4da-c57bdac6a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:55.096558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6179d-9b61-455a-a4da-c57bdac6a390 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:55.096852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6179d-9b61-455a-a4da-c57bdac6a390 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:55.097417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6179d-9b61-455a-a4da-c57bdac6a390 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:55.099621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6179d-9b61-455a-a4da-c57bdac6a390 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:55.101540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6179d-9b61-455a-a4da-c57bdac6a390 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:55.104476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 102/206] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:54 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:55.110567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6179d-9b61-455a-a4da-c57bdac6a390 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:55.147548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7935b2-9fcd-4a84-b5ed-e7466a1e093f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:55.189783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e91a2-acc8-4513-b7bf-26abadb7de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:55.196382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e91a2-acc8-4513-b7bf-26abadb7de2a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:55.196382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e91a2-acc8-4513-b7bf-26abadb7de2a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:55.196382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e91a2-acc8-4513-b7bf-26abadb7de2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:55.196382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e91a2-acc8-4513-b7bf-26abadb7de2a None admin] RBAC: Adding query filter params (role.id=47878794de344232a226a95ce5771504, scope.project.id=7f770973effe4795978f1b5d9c097b92, user.id=8d8f4077ceea4670a530f88d68330e4e) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:55.196382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e91a2-acc8-4513-b7bf-26abadb7de2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:55.196382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e91a2-acc8-4513-b7bf-26abadb7de2a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:55.201499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 105/207] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:55 2022] GET /identity/v3/role_assignments?user.id=8d8f4077ceea4670a530f88d68330e4e&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=47878794de344232a226a95ce5771504 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:55.211406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:55.249053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6179d-9b61-455a-a4da-c57bdac6a390 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:55.289337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611322a2-293a-4f03-bdd4-28be402d6f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:55.290372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611322a2-293a-4f03-bdd4-28be402d6f9a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:55.290897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611322a2-293a-4f03-bdd4-28be402d6f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:55.291376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611322a2-293a-4f03-bdd4-28be402d6f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:55.292413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611322a2-293a-4f03-bdd4-28be402d6f9a None admin] RBAC: Adding query filter params (role.id=c78dd2dd41224a73aec3edfe5d103a84, scope.project.id=822c049cabcb4edb8210aa9424fbb06a, user.id=27352662136f47c8845c3766faf50315) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:55.292958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611322a2-293a-4f03-bdd4-28be402d6f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:55.294694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611322a2-293a-4f03-bdd4-28be402d6f9a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:55.303645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 103/208] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:55 2022] GET /identity/v3/role_assignments?user.id=27352662136f47c8845c3766faf50315&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=c78dd2dd41224a73aec3edfe5d103a84 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:55.986835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6fbca-c702-40f1-ae0f-bf11de6b7b9b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:55.987334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6fbca-c702-40f1-ae0f-bf11de6b7b9b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:55.987436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6fbca-c702-40f1-ae0f-bf11de6b7b9b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:55.990556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 106/209] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:55.995980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4049b8-d2c5-4097-bc1a-29c7b00df16d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:55.996295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4049b8-d2c5-4097-bc1a-29c7b00df16d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:55.996569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4049b8-d2c5-4097-bc1a-29c7b00df16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:56.018337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4d4049b8-d2c5-4097-bc1a-29c7b00df16d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:56.019470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4d4049b8-d2c5-4097-bc1a-29c7b00df16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:56.084348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef097b-9b1e-48ca-8742-5a8d1f163765 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:56.084760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef097b-9b1e-48ca-8742-5a8d1f163765 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:56.084861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef097b-9b1e-48ca-8742-5a8d1f163765 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:56.086492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 107/210] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:56.091878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25ff9-62ef-4a7d-b346-b1e2faba13a3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:56.092330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25ff9-62ef-4a7d-b346-b1e2faba13a3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:56.092590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d25ff9-62ef-4a7d-b346-b1e2faba13a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:56.096302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 104/211] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:56.114397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-93d25ff9-62ef-4a7d-b346-b1e2faba13a3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:56.115987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-93d25ff9-62ef-4a7d-b346-b1e2faba13a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:56.189722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 108/212] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:56.381569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4049b8-d2c5-4097-bc1a-29c7b00df16d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:56.455772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:56.471168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d25ff9-62ef-4a7d-b346-b1e2faba13a3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:56.497262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:56.506551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4049b8-d2c5-4097-bc1a-29c7b00df16d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:56.543690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c65eb-2af7-4df7-86d5-16bd93366c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:56.544333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c65eb-2af7-4df7-86d5-16bd93366c32 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:56.544614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c65eb-2af7-4df7-86d5-16bd93366c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:56.544881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c65eb-2af7-4df7-86d5-16bd93366c32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:56.556634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c65eb-2af7-4df7-86d5-16bd93366c32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d43c65eb-2af7-4df7-86d5-16bd93366c32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:56.561946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:56.562848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:56.563797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 105/213] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:56.568968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c65eb-2af7-4df7-86d5-16bd93366c32 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:56.603340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d25ff9-62ef-4a7d-b346-b1e2faba13a3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:56.648092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313ec36-7e31-485d-9a45-698c8bf19498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:56.649643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313ec36-7e31-485d-9a45-698c8bf19498 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:56.650417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313ec36-7e31-485d-9a45-698c8bf19498 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:56.651219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313ec36-7e31-485d-9a45-698c8bf19498 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:56.653648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313ec36-7e31-485d-9a45-698c8bf19498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:56.655670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313ec36-7e31-485d-9a45-698c8bf19498 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:56.657569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 109/214] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:17:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:56.664637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313ec36-7e31-485d-9a45-698c8bf19498 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:56.697288 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c65eb-2af7-4df7-86d5-16bd93366c32 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:56.724567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:56.736928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:56.737905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:56.738423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:56.739088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:56.740098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:56.740644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:56.742542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:56.762099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 106/215] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:17:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:56.763558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:56.780798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:56.799046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313ec36-7e31-485d-9a45-698c8bf19498 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:56.844527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f630add-3781-41c9-9156-de2a30b5718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:56.845255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630add-3781-41c9-9156-de2a30b5718f None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:56.845566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630add-3781-41c9-9156-de2a30b5718f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:56.845939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f630add-3781-41c9-9156-de2a30b5718f None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:56.846615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f630add-3781-41c9-9156-de2a30b5718f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:56.848645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f630add-3781-41c9-9156-de2a30b5718f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:56.892809 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d8c4e-6f22-473f-9ec3-c4dd32009312 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:56.931991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da47ee-4f6b-4de4-a658-09a19200afda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:56.932634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da47ee-4f6b-4de4-a658-09a19200afda None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:56.932978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da47ee-4f6b-4de4-a658-09a19200afda None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:56.933330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da47ee-4f6b-4de4-a658-09a19200afda None admin] PATH_INFO: `/v3/users/cbd65a09d6c945449b9fe2179aa94b00` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:56.949277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da47ee-4f6b-4de4-a658-09a19200afda None admin] RBAC: Authorizing `identity:get_user(user_id=cbd65a09d6c945449b9fe2179aa94b00)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:56.950862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da47ee-4f6b-4de4-a658-09a19200afda None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:56.958061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 107/216] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:56 2022] GET /identity/v3/users/cbd65a09d6c945449b9fe2179aa94b00 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:56.959087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da47ee-4f6b-4de4-a658-09a19200afda None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:56.971883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 110/217] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:17:56 2022] POST /identity/v3/users => generated 297 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:17:57.032570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:57.106734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da47ee-4f6b-4de4-a658-09a19200afda None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:57.149245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b45e5-8277-4cb6-9ff6-66bea0fa223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:57.149983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b45e5-8277-4cb6-9ff6-66bea0fa223a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:57.150263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b45e5-8277-4cb6-9ff6-66bea0fa223a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:57.150533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b45e5-8277-4cb6-9ff6-66bea0fa223a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:57.205536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b0cf5-5aaa-4a08-8fcc-2dfecc84f18e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:57.206267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b0cf5-5aaa-4a08-8fcc-2dfecc84f18e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:57.206942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b0cf5-5aaa-4a08-8fcc-2dfecc84f18e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:57.210730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 111/218] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:57.219546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d1b3d-bb9f-40b9-83db-96f7aba53632 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:57.219787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d1b3d-bb9f-40b9-83db-96f7aba53632 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:57.220229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d1b3d-bb9f-40b9-83db-96f7aba53632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:57.252523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cf7d1b3d-bb9f-40b9-83db-96f7aba53632 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:57.256334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cf7d1b3d-bb9f-40b9-83db-96f7aba53632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:57.258981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b45e5-8277-4cb6-9ff6-66bea0fa223a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:57.263186 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b45e5-8277-4cb6-9ff6-66bea0fa223a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:57.345576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:57.365080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 112/219] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:57.374806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e822cf-3281-40ce-977d-c20d4fe76b1d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:57.375134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e822cf-3281-40ce-977d-c20d4fe76b1d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:57.375366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e822cf-3281-40ce-977d-c20d4fe76b1d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:57.377268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 113/220] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:17:57.384381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ca18-554c-410e-b8ed-0a99cd9a6da3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:57.384694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ca18-554c-410e-b8ed-0a99cd9a6da3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:57.384943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ca18-554c-410e-b8ed-0a99cd9a6da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:57.405251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6315ca18-554c-410e-b8ed-0a99cd9a6da3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:57.406500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6315ca18-554c-410e-b8ed-0a99cd9a6da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:57.423232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 108/221] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:17:57.433245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b45e5-8277-4cb6-9ff6-66bea0fa223a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:57.501166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 114/222] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:17:57.553860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b45e5-8277-4cb6-9ff6-66bea0fa223a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:57.597981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369fb26-bf45-4c72-bff4-e79979717d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:57.598668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369fb26-bf45-4c72-bff4-e79979717d38 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:57.598941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369fb26-bf45-4c72-bff4-e79979717d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:57.599267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369fb26-bf45-4c72-bff4-e79979717d38 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:57.600467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369fb26-bf45-4c72-bff4-e79979717d38 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:57.601942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369fb26-bf45-4c72-bff4-e79979717d38 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:57.603605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 109/223] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:57 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:57.609909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ca18-554c-410e-b8ed-0a99cd9a6da3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:57.624738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:57.713359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369fb26-bf45-4c72-bff4-e79979717d38 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:57.730672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:57.748220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ca18-554c-410e-b8ed-0a99cd9a6da3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:57.813961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:57.814575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:57.814853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:57.815124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:57.815860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] RBAC: Adding query filter params (role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=7f770973effe4795978f1b5d9c097b92, user.id=cbd65a09d6c945449b9fe2179aa94b00) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:57.816166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:57.819141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:57.825507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 115/224] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:17:57 2022] GET /identity/v3/role_assignments?user.id=cbd65a09d6c945449b9fe2179aa94b00&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:57.826406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:57.927655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369fb26-bf45-4c72-bff4-e79979717d38 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:57.976122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be8ee4-ace3-4554-920a-4991e7709007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:57.976801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8ee4-ace3-4554-920a-4991e7709007 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:57.977084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8ee4-ace3-4554-920a-4991e7709007 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:57.977359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be8ee4-ace3-4554-920a-4991e7709007 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:57.996921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:58.003303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be8ee4-ace3-4554-920a-4991e7709007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c2be8ee4-ace3-4554-920a-4991e7709007 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:58.009602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:58.010549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:17:58.011400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:17:58.013957 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 110/225] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:17:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:17:58.020047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be8ee4-ace3-4554-920a-4991e7709007 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:58.059573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974395a-7568-420d-bc20-43a80d07ba6a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:58.104880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eae63-282a-45c0-8ede-051273ab6707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:58.105437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eae63-282a-45c0-8ede-051273ab6707 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:58.105723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eae63-282a-45c0-8ede-051273ab6707 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:58.106000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082eae63-282a-45c0-8ede-051273ab6707 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:58.118705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082eae63-282a-45c0-8ede-051273ab6707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-082eae63-282a-45c0-8ede-051273ab6707 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:17:58.124136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:17:58.125069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:17:58.125850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 116/226] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:17:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:17:58.131411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eae63-282a-45c0-8ede-051273ab6707 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:58.136571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:58.173410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be8ee4-ace3-4554-920a-4991e7709007 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:58.219196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:58.220066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:58.220399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:58.220675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:58.221393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:58.221731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:58.224880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:58.239629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 111/227] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:17:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:58.248907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:58.322479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082eae63-282a-45c0-8ede-051273ab6707 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:58.364878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:58.372728 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:58.373352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:58.375944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:58.375944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:58.375944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:17:58.375944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:58.376356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:58.386272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ebddb-ada8-4664-adee-5cb3ddec3fca None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:58.386875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 117/228] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:17:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:58.392471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:58.426978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbbaaf-aad3-4e58-b9a2-df700c607817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:58.427671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbbaaf-aad3-4e58-b9a2-df700c607817 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:58.427956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbbaaf-aad3-4e58-b9a2-df700c607817 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:58.428256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbbaaf-aad3-4e58-b9a2-df700c607817 None admin] PATH_INFO: `/v3/users/8d8f4077ceea4670a530f88d68330e4e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:58.429601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbbaaf-aad3-4e58-b9a2-df700c607817 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8f4077ceea4670a530f88d68330e4e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:58.431207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbbaaf-aad3-4e58-b9a2-df700c607817 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:58.433361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 112/229] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:58 2022] GET /identity/v3/users/8d8f4077ceea4670a530f88d68330e4e => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:17:58.438983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbbaaf-aad3-4e58-b9a2-df700c607817 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:58.442472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:58.499491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061ecea-5196-47d0-81ff-9f2b2df7edb6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:58.536685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226afc6-2246-4bc0-be5b-b780f818533b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:58.537326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226afc6-2246-4bc0-be5b-b780f818533b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:58.537616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226afc6-2246-4bc0-be5b-b780f818533b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:58.537893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226afc6-2246-4bc0-be5b-b780f818533b None admin] PATH_INFO: `/v3/users/8d8f4077ceea4670a530f88d68330e4e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:58.542301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226afc6-2246-4bc0-be5b-b780f818533b None admin] RBAC: Authorizing `identity:get_user(user_id=8d8f4077ceea4670a530f88d68330e4e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:58.543796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226afc6-2246-4bc0-be5b-b780f818533b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:58.550741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 118/230] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:17:58 2022] GET /identity/v3/users/8d8f4077ceea4670a530f88d68330e4e => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:58.557083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226afc6-2246-4bc0-be5b-b780f818533b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:58.575143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbbaaf-aad3-4e58-b9a2-df700c607817 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:58.654934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97618e44-15f7-486f-b2bb-5de6ab5022e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:58.655915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97618e44-15f7-486f-b2bb-5de6ab5022e4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:58.656446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97618e44-15f7-486f-b2bb-5de6ab5022e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:58.656940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97618e44-15f7-486f-b2bb-5de6ab5022e4 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:58.662338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97618e44-15f7-486f-b2bb-5de6ab5022e4 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:58.665190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97618e44-15f7-486f-b2bb-5de6ab5022e4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:58.668641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 113/231] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:58 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:58.675078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97618e44-15f7-486f-b2bb-5de6ab5022e4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:58.693746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:58.755921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:58.787533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226afc6-2246-4bc0-be5b-b780f818533b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:58.791781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97618e44-15f7-486f-b2bb-5de6ab5022e4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:58.870239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafda82c-f2c0-41d5-905d-7ea6d61514fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:58.870916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafda82c-f2c0-41d5-905d-7ea6d61514fc None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:58.871195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafda82c-f2c0-41d5-905d-7ea6d61514fc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:58.871469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafda82c-f2c0-41d5-905d-7ea6d61514fc None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92/users/8d8f4077ceea4670a530f88d68330e4e/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:58.874261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafda82c-f2c0-41d5-905d-7ea6d61514fc None admin] RBAC: Authorizing `identity:create_grant(project_id=7f770973effe4795978f1b5d9c097b92, user_id=8d8f4077ceea4670a530f88d68330e4e, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:58.877232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafda82c-f2c0-41d5-905d-7ea6d61514fc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:58.881418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568b835-290c-4ac8-8358-1f2d1379349d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:58.882060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568b835-290c-4ac8-8358-1f2d1379349d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:58.882337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568b835-290c-4ac8-8358-1f2d1379349d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:58.882607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568b835-290c-4ac8-8358-1f2d1379349d None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:58.885011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568b835-290c-4ac8-8358-1f2d1379349d None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:58.886482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568b835-290c-4ac8-8358-1f2d1379349d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:58.889352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 119/232] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:17:58 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:17:58.895120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568b835-290c-4ac8-8358-1f2d1379349d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:17:58.971223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 114/233] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:17:58 2022] PUT /identity/v3/projects/7f770973effe4795978f1b5d9c097b92/users/8d8f4077ceea4670a530f88d68330e4e/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:17:59.041771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568b835-290c-4ac8-8358-1f2d1379349d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:17:59.079744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:17:59.090059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfd82a-af67-4307-83f0-101915cd38d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:17:59.090812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd82a-af67-4307-83f0-101915cd38d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:59.091103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd82a-af67-4307-83f0-101915cd38d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:59.091380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd82a-af67-4307-83f0-101915cd38d0 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92/users/8d8f4077ceea4670a530f88d68330e4e/roles/47878794de344232a226a95ce5771504` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:59.094341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfd82a-af67-4307-83f0-101915cd38d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f770973effe4795978f1b5d9c097b92, user_id=8d8f4077ceea4670a530f88d68330e4e, role_id=47878794de344232a226a95ce5771504)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:17:59.095887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfd82a-af67-4307-83f0-101915cd38d0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:17:59.196426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 120/234] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:17:58 2022] PUT /identity/v3/projects/7f770973effe4795978f1b5d9c097b92/users/8d8f4077ceea4670a530f88d68330e4e/roles/47878794de344232a226a95ce5771504 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:17:59.506275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b485-22d3-4d1d-a55b-4fc1bf7d528e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:59.506786 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b485-22d3-4d1d-a55b-4fc1bf7d528e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:59.506891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b485-22d3-4d1d-a55b-4fc1bf7d528e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:59.508526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 115/235] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:59.514129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07c4d-b089-4ab8-a4f5-9fb9661e848d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:59.514558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07c4d-b089-4ab8-a4f5-9fb9661e848d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:59.514731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a07c4d-b089-4ab8-a4f5-9fb9661e848d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:59.544213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-63a07c4d-b089-4ab8-a4f5-9fb9661e848d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:17:59.545587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-63a07c4d-b089-4ab8-a4f5-9fb9661e848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:17:59.641790 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 121/236] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:17:59.967003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19e21-f0e6-403d-9a4e-57f0b438ea24 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:59.967746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19e21-f0e6-403d-9a4e-57f0b438ea24 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:59.968262 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19e21-f0e6-403d-9a4e-57f0b438ea24 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:17:59.970203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 116/237] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:17:59.977101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe493b8-a2e1-43f6-ad1b-b7ffff5ed538 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:17:59.977511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe493b8-a2e1-43f6-ad1b-b7ffff5ed538 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:17:59.977913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe493b8-a2e1-43f6-ad1b-b7ffff5ed538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:00.017871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-abe493b8-a2e1-43f6-ad1b-b7ffff5ed538 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:00.020322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-abe493b8-a2e1-43f6-ad1b-b7ffff5ed538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:00.119173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b19e21-f0e6-403d-9a4e-57f0b438ea24 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:00.171629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 122/238] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:00.273858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b19e21-f0e6-403d-9a4e-57f0b438ea24 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:00.304331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:00.318554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2eb6c3-90a9-4831-bacd-1fb8597a9800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:00.319199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eb6c3-90a9-4831-bacd-1fb8597a9800 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:00.319527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eb6c3-90a9-4831-bacd-1fb8597a9800 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:00.319802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2eb6c3-90a9-4831-bacd-1fb8597a9800 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:00.331512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2eb6c3-90a9-4831-bacd-1fb8597a9800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-0e2eb6c3-90a9-4831-bacd-1fb8597a9800 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:00.337384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:00.338303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:00.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 117/239] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:00.344438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe493b8-a2e1-43f6-ad1b-b7ffff5ed538 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:00.353476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b8ad9-3ef2-4cd4-84a9-8e9648126ce2 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:00.353780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b8ad9-3ef2-4cd4-84a9-8e9648126ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:00.354024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b8ad9-3ef2-4cd4-84a9-8e9648126ce2 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:00.355696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 118/240] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:00.360890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08dae5-8d4a-4031-a4e2-91da9ac3894f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:00.361169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08dae5-8d4a-4031-a4e2-91da9ac3894f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:00.361442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08dae5-8d4a-4031-a4e2-91da9ac3894f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:00.383039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8d08dae5-8d4a-4031-a4e2-91da9ac3894f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:00.384254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8d08dae5-8d4a-4031-a4e2-91da9ac3894f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:00.448656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:00.494544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 119/241] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:00.501934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08dae5-8d4a-4031-a4e2-91da9ac3894f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:00.514932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe493b8-a2e1-43f6-ad1b-b7ffff5ed538 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:00.565017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:00.566313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:00.567619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:00.570292 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:00.571259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:00.571767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:00.574221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:00.598609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 123/242] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:00.601931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:00.716859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08dae5-8d4a-4031-a4e2-91da9ac3894f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:00.765327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:00.805843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bce8d3-47f5-4a67-9b6a-5ce5e7a3691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:00.806561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bce8d3-47f5-4a67-9b6a-5ce5e7a3691b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:00.806838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bce8d3-47f5-4a67-9b6a-5ce5e7a3691b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:00.807110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bce8d3-47f5-4a67-9b6a-5ce5e7a3691b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:00.809459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bce8d3-47f5-4a67-9b6a-5ce5e7a3691b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:00.810990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bce8d3-47f5-4a67-9b6a-5ce5e7a3691b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:00.812950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 120/243] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:00.818785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bce8d3-47f5-4a67-9b6a-5ce5e7a3691b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:00.841754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55119c57-56a4-4939-a016-018e2b7a6815 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:00.854248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:00.888422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e0507-51db-4e6e-8563-1c7c3aaa12b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:00.889472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0507-51db-4e6e-8563-1c7c3aaa12b4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:00.889958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0507-51db-4e6e-8563-1c7c3aaa12b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:00.890429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0507-51db-4e6e-8563-1c7c3aaa12b4 None admin] PATH_INFO: `/v3/users/ff4691d8ccdf46a6a7842f51877feebf` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:00.907620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e0507-51db-4e6e-8563-1c7c3aaa12b4 None admin] RBAC: Authorizing `identity:get_user(user_id=ff4691d8ccdf46a6a7842f51877feebf)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:00.909593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e0507-51db-4e6e-8563-1c7c3aaa12b4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:00.913551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 124/244] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:00 2022] GET /identity/v3/users/ff4691d8ccdf46a6a7842f51877feebf => generated 297 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:00.914848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e0507-51db-4e6e-8563-1c7c3aaa12b4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:00.939231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bce8d3-47f5-4a67-9b6a-5ce5e7a3691b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:00.978654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce03bd-1cf4-4402-939b-27d8754a9487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:00.979299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce03bd-1cf4-4402-939b-27d8754a9487 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:00.979576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce03bd-1cf4-4402-939b-27d8754a9487 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:00.979845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce03bd-1cf4-4402-939b-27d8754a9487 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:00.991476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce03bd-1cf4-4402-939b-27d8754a9487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-f3ce03bd-1cf4-4402-939b-27d8754a9487 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:00.996389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:00.997299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:00.998093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 121/245] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:00.998749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce03bd-1cf4-4402-939b-27d8754a9487 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:01.040404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e0507-51db-4e6e-8563-1c7c3aaa12b4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:01.051647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:01.081639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f1cec-c81c-4545-b8e3-b773e7f6d243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:01.083178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1cec-c81c-4545-b8e3-b773e7f6d243 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.083662 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1cec-c81c-4545-b8e3-b773e7f6d243 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.084116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f1cec-c81c-4545-b8e3-b773e7f6d243 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.084978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f1cec-c81c-4545-b8e3-b773e7f6d243 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:01.086631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f1cec-c81c-4545-b8e3-b773e7f6d243 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:01.107760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce03bd-1cf4-4402-939b-27d8754a9487 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:01.148160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebec88-c339-4ba1-810b-99773723aee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:01.148793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebec88-c339-4ba1-810b-99773723aee5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.149067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebec88-c339-4ba1-810b-99773723aee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.149334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebec88-c339-4ba1-810b-99773723aee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.163040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 125/246] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:18:00 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:18:01.164507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f1cec-c81c-4545-b8e3-b773e7f6d243 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:01.200798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:01.306856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f1cec-c81c-4545-b8e3-b773e7f6d243 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:01.330067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebec88-c339-4ba1-810b-99773723aee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:01.331498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebec88-c339-4ba1-810b-99773723aee5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:01.352698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:01.353671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.354204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.355324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.356298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:01.356857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:01.358657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:01.370249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 126/247] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:01.375895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:01.445715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:01.499760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:01.513170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 122/248] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:01.514476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29173f-a625-4d9c-af46-342a465cdf1f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.514762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29173f-a625-4d9c-af46-342a465cdf1f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.515038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29173f-a625-4d9c-af46-342a465cdf1f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.516469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 123/249] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:01.518648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ab698-d781-4564-86b2-559847525333 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.518963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ab698-d781-4564-86b2-559847525333 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.520144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ab698-d781-4564-86b2-559847525333 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.521236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 124/250] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:01.521958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ab698-d781-4564-86b2-559847525333 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:01.523263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7893fff-d5f2-4977-b7c9-fb6d00461189 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:01.571139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e411af-4c3d-45d0-b189-13a10e3a9275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:01.571793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e411af-4c3d-45d0-b189-13a10e3a9275 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.572078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e411af-4c3d-45d0-b189-13a10e3a9275 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.572426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e411af-4c3d-45d0-b189-13a10e3a9275 None admin] PATH_INFO: `/v3/users/cbd65a09d6c945449b9fe2179aa94b00` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.573649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e411af-4c3d-45d0-b189-13a10e3a9275 None admin] RBAC: Authorizing `identity:get_user(user_id=cbd65a09d6c945449b9fe2179aa94b00)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:01.575251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e411af-4c3d-45d0-b189-13a10e3a9275 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:01.580353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 127/251] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:01 2022] GET /identity/v3/users/cbd65a09d6c945449b9fe2179aa94b00 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:01.582234 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110177-a81d-4b49-87a4-d88ad93eeb26 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.582607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110177-a81d-4b49-87a4-d88ad93eeb26 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.582904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62110177-a81d-4b49-87a4-d88ad93eeb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.607582 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-62110177-a81d-4b49-87a4-d88ad93eeb26 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:01.608414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-62110177-a81d-4b49-87a4-d88ad93eeb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:01.661774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ab698-d781-4564-86b2-559847525333 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:01.700116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 128/252] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:01.701419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649b7dd-9d71-4f55-98bc-98da5fcb6515 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.702073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649b7dd-9d71-4f55-98bc-98da5fcb6515 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.702365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649b7dd-9d71-4f55-98bc-98da5fcb6515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.710266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b9b6b-cb06-44dc-9f94-f229857ee9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:01.711030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b9b6b-cb06-44dc-9f94-f229857ee9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:01.711306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b9b6b-cb06-44dc-9f94-f229857ee9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:01.711598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b9b6b-cb06-44dc-9f94-f229857ee9e1 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:01.712807 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b9b6b-cb06-44dc-9f94-f229857ee9e1 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:01.714800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b9b6b-cb06-44dc-9f94-f229857ee9e1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:01.720210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 125/253] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:01 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:01.720210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b9b6b-cb06-44dc-9f94-f229857ee9e1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:01.730182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a649b7dd-9d71-4f55-98bc-98da5fcb6515 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:01.732350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a649b7dd-9d71-4f55-98bc-98da5fcb6515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:01.744269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:01.839275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 129/254] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:01.840478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649b7dd-9d71-4f55-98bc-98da5fcb6515 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:01.875647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:01.951020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b9b6b-cb06-44dc-9f94-f229857ee9e1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:02.007196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649b7dd-9d71-4f55-98bc-98da5fcb6515 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:02.030298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95faa96-99ab-4666-9741-1d7d59916e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:02.030952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95faa96-99ab-4666-9741-1d7d59916e54 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:02.031238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95faa96-99ab-4666-9741-1d7d59916e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:02.031515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95faa96-99ab-4666-9741-1d7d59916e54 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:02.032769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95faa96-99ab-4666-9741-1d7d59916e54 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:02.034237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95faa96-99ab-4666-9741-1d7d59916e54 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:02.036912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 126/255] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:01 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:02.042734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95faa96-99ab-4666-9741-1d7d59916e54 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:02.077837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:02.078458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:02.078735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:02.079003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:02.079741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] RBAC: Adding query filter params (role.id=c78dd2dd41224a73aec3edfe5d103a84, scope.project.id=7f770973effe4795978f1b5d9c097b92, user.id=ff4691d8ccdf46a6a7842f51877feebf) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:02.080056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:02.081542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:02.095139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 130/256] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:01 2022] GET /identity/v3/role_assignments?user.id=ff4691d8ccdf46a6a7842f51877feebf&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=c78dd2dd41224a73aec3edfe5d103a84 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:02.197960 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:02.227051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95faa96-99ab-4666-9741-1d7d59916e54 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:02.231255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:02.295251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5e2c7-dd18-4cfb-b756-27a836f4a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:02.295935 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e2c7-dd18-4cfb-b756-27a836f4a0c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:02.296238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e2c7-dd18-4cfb-b756-27a836f4a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:02.296521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e2c7-dd18-4cfb-b756-27a836f4a0c0 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92/users/cbd65a09d6c945449b9fe2179aa94b00/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:02.304213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5e2c7-dd18-4cfb-b756-27a836f4a0c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f770973effe4795978f1b5d9c097b92, user_id=cbd65a09d6c945449b9fe2179aa94b00, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:02.305824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5e2c7-dd18-4cfb-b756-27a836f4a0c0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:02.351990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:02.378734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebea464-436a-4c5d-bd1d-3e3f19d31912 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:02.396233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 127/257] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:18:02 2022] PUT /identity/v3/projects/7f770973effe4795978f1b5d9c097b92/users/cbd65a09d6c945449b9fe2179aa94b00/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:02.397402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5e2c7-dd18-4cfb-b756-27a836f4a0c0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:02.444705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff23ead-a0ff-4fc0-b4e4-96b2570d9e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:02.445416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23ead-a0ff-4fc0-b4e4-96b2570d9e41 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:02.445701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23ead-a0ff-4fc0-b4e4-96b2570d9e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:02.445979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff23ead-a0ff-4fc0-b4e4-96b2570d9e41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:02.462999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff23ead-a0ff-4fc0-b4e4-96b2570d9e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-0ff23ead-a0ff-4fc0-b4e4-96b2570d9e41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:02.470833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:02.472036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:02.473097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 131/258] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:18:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:02.479362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff23ead-a0ff-4fc0-b4e4-96b2570d9e41 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:02.627994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff23ead-a0ff-4fc0-b4e4-96b2570d9e41 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:02.631388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5e2c7-dd18-4cfb-b756-27a836f4a0c0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:02.706343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:02.707514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:02.707852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:02.708273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:02.708955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:02.709670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:02.711745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:02.720647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:02.723963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 132/259] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:02.729371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:02.734731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a548-d131-4d7c-89cd-9c714ef6e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:02.735380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a548-d131-4d7c-89cd-9c714ef6e167 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:02.740236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a548-d131-4d7c-89cd-9c714ef6e167 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:02.740236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5a548-d131-4d7c-89cd-9c714ef6e167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:02.780884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5a548-d131-4d7c-89cd-9c714ef6e167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e1f5a548-d131-4d7c-89cd-9c714ef6e167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:02.786284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:02.787213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:02.788012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 128/260] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:18:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:02.813259 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a548-d131-4d7c-89cd-9c714ef6e167 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:02.825799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:02.855050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58075fe-9b12-4fa7-927a-feca07bc65e2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:02.929909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f520af-5c66-4c79-8cac-8d884dfef3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:02.930562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f520af-5c66-4c79-8cac-8d884dfef3cc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:02.930841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f520af-5c66-4c79-8cac-8d884dfef3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:02.931112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f520af-5c66-4c79-8cac-8d884dfef3cc None admin] PATH_INFO: `/v3/users/8d8f4077ceea4670a530f88d68330e4e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:02.932948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f520af-5c66-4c79-8cac-8d884dfef3cc None admin] RBAC: Authorizing `identity:get_user(user_id=8d8f4077ceea4670a530f88d68330e4e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:02.934677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f520af-5c66-4c79-8cac-8d884dfef3cc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:02.936552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 133/261] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:02 2022] GET /identity/v3/users/8d8f4077ceea4670a530f88d68330e4e => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:02.942871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f520af-5c66-4c79-8cac-8d884dfef3cc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:03.000012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5a548-d131-4d7c-89cd-9c714ef6e167 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:03.036393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:03.037009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:03.037290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:03.037597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:03.038299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:03.038602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:03.040015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:03.049859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 129/262] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:03.060237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:03.120536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:03.163990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:03.212045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d90ab-5aad-4bb2-8820-6c832026c204 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:03.214517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f520af-5c66-4c79-8cac-8d884dfef3cc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:03.263715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fe3af-f611-476c-912b-bb8cbd3b75d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:03.264296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fe3af-f611-476c-912b-bb8cbd3b75d2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:03.264552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fe3af-f611-476c-912b-bb8cbd3b75d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:03.264826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fe3af-f611-476c-912b-bb8cbd3b75d2 None admin] PATH_INFO: `/v3/users/8d8f4077ceea4670a530f88d68330e4e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:03.265975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fe3af-f611-476c-912b-bb8cbd3b75d2 None admin] RBAC: Authorizing `identity:get_user(user_id=8d8f4077ceea4670a530f88d68330e4e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:03.267431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fe3af-f611-476c-912b-bb8cbd3b75d2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:03.273000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 130/263] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:03 2022] GET /identity/v3/users/8d8f4077ceea4670a530f88d68330e4e => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:03.274843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fe3af-f611-476c-912b-bb8cbd3b75d2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:03.288056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8294d6-a451-404b-ac1b-baaa07ebf020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:03.289055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8294d6-a451-404b-ac1b-baaa07ebf020 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:03.289534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8294d6-a451-404b-ac1b-baaa07ebf020 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:03.290003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8294d6-a451-404b-ac1b-baaa07ebf020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:03.436527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8294d6-a451-404b-ac1b-baaa07ebf020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:03.438437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8294d6-a451-404b-ac1b-baaa07ebf020 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:03.473194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fe3af-f611-476c-912b-bb8cbd3b75d2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:03.536203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860862fc-376e-444b-8da1-e1139bd13094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:03.536203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860862fc-376e-444b-8da1-e1139bd13094 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:03.536203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860862fc-376e-444b-8da1-e1139bd13094 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:03.536203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860862fc-376e-444b-8da1-e1139bd13094 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:03.538743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:03.620566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:03.648644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860862fc-376e-444b-8da1-e1139bd13094 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:03.650122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860862fc-376e-444b-8da1-e1139bd13094 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:03.697588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 134/264] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:18:03.703453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8294d6-a451-404b-ac1b-baaa07ebf020 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:03.835905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 131/265] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:03.839220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860862fc-376e-444b-8da1-e1139bd13094 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:03.912902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:03.933341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8294d6-a451-404b-ac1b-baaa07ebf020 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:03.992760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0807c-01aa-4c54-8906-ba6c01fcadbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:03.993403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0807c-01aa-4c54-8906-ba6c01fcadbf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:03.993686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0807c-01aa-4c54-8906-ba6c01fcadbf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:03.993961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0807c-01aa-4c54-8906-ba6c01fcadbf None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:03.995137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0807c-01aa-4c54-8906-ba6c01fcadbf None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:03.996642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0807c-01aa-4c54-8906-ba6c01fcadbf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:03.998741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 135/266] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:03 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:04.005127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0807c-01aa-4c54-8906-ba6c01fcadbf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:04.042933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:04.056370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860862fc-376e-444b-8da1-e1139bd13094 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:04.117759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0807c-01aa-4c54-8906-ba6c01fcadbf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:04.119794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5aa974-a782-4897-b984-3d8febb5f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:04.120597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5aa974-a782-4897-b984-3d8febb5f338 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.121198 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5aa974-a782-4897-b984-3d8febb5f338 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.121680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5aa974-a782-4897-b984-3d8febb5f338 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.125065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5aa974-a782-4897-b984-3d8febb5f338 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:04.127251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5aa974-a782-4897-b984-3d8febb5f338 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:04.129358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 132/267] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:03 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:04.130850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76fb3-1bd8-4b13-a289-ac980134056e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.131139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76fb3-1bd8-4b13-a289-ac980134056e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.131456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c76fb3-1bd8-4b13-a289-ac980134056e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.133216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 133/268] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:04.142076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef0f9a-f250-4ca9-a77a-66d73be8048c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.142381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef0f9a-f250-4ca9-a77a-66d73be8048c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.142644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef0f9a-f250-4ca9-a77a-66d73be8048c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.177280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:04.177939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.178226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.178502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.179295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] RBAC: Adding query filter params (role.id=47878794de344232a226a95ce5771504, scope.project.id=7f770973effe4795978f1b5d9c097b92, user.id=8d8f4077ceea4670a530f88d68330e4e) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:04.179609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:04.181255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:04.183579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a6ef0f9a-f250-4ca9-a77a-66d73be8048c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:04.184890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a6ef0f9a-f250-4ca9-a77a-66d73be8048c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:04.198912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 136/269] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:04 2022] GET /identity/v3/role_assignments?user.id=8d8f4077ceea4670a530f88d68330e4e&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=47878794de344232a226a95ce5771504 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:04.199836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:04.393158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 134/270] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:04.449824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:04.454650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3891e42-a5a0-4dfc-9354-bbab2cd9b47f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:04.460700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f893-e5e9-449f-bc15-6d02d62dce14 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.461324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f893-e5e9-449f-bc15-6d02d62dce14 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.466990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f893-e5e9-449f-bc15-6d02d62dce14 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.466990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 135/271] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:04.469180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2cbb7-0338-4e20-b74b-aae452e97e0c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.469686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2cbb7-0338-4e20-b74b-aae452e97e0c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.470159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2cbb7-0338-4e20-b74b-aae452e97e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.497614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:04.500778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.501279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.501862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.502842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=7f770973effe4795978f1b5d9c097b92, user.id=8d8f4077ceea4670a530f88d68330e4e) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:04.504871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-49f2cbb7-0338-4e20-b74b-aae452e97e0c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:04.505424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:04.508402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-49f2cbb7-0338-4e20-b74b-aae452e97e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:04.508690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:04.522574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 137/272] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:04 2022] GET /identity/v3/role_assignments?user.id=8d8f4077ceea4670a530f88d68330e4e&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 608 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:04.549229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:04.613172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 136/273] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:04.811510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:04.945692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45005728-40db-407d-8f58-6fb9ac2f4651 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:04.969862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc41f6c-864d-4b5a-9302-b9a1e355c7c2 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.970389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc41f6c-864d-4b5a-9302-b9a1e355c7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.970389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc41f6c-864d-4b5a-9302-b9a1e355c7c2 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.971846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 137/274] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:04.977123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de853aa-abf3-4c62-859e-47dc21f0bdba None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.977410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de853aa-abf3-4c62-859e-47dc21f0bdba None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.977686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de853aa-abf3-4c62-859e-47dc21f0bdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.987449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99ad22-b46c-4191-a7ca-775d03aac5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:04.988798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ad22-b46c-4191-a7ca-775d03aac5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:04.989129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ad22-b46c-4191-a7ca-775d03aac5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:04.989395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99ad22-b46c-4191-a7ca-775d03aac5f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:04.997581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5de853aa-abf3-4c62-859e-47dc21f0bdba None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:04.998846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5de853aa-abf3-4c62-859e-47dc21f0bdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:05.005777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99ad22-b46c-4191-a7ca-775d03aac5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ff99ad22-b46c-4191-a7ca-775d03aac5f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:05.012805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:05.013910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:05.014866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:05.020478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 138/275] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:05.025712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99ad22-b46c-4191-a7ca-775d03aac5f7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.058399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:05.086473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 138/276] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:05.092220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de853aa-abf3-4c62-859e-47dc21f0bdba None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.211643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99ad22-b46c-4191-a7ca-775d03aac5f7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:05.258539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b958b3a-a0c0-4591-a473-cdc2471013a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:05.259905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958b3a-a0c0-4591-a473-cdc2471013a7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.260208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958b3a-a0c0-4591-a473-cdc2471013a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.260501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b958b3a-a0c0-4591-a473-cdc2471013a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.261192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b958b3a-a0c0-4591-a473-cdc2471013a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:05.261498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b958b3a-a0c0-4591-a473-cdc2471013a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:05.263328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b958b3a-a0c0-4591-a473-cdc2471013a7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:05.278188 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 139/277] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:05.279621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51d754-94e3-468b-97cd-d9e6560051e7 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.280016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51d754-94e3-468b-97cd-d9e6560051e7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.280255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51d754-94e3-468b-97cd-d9e6560051e7 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.281554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 140/278] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:05.282428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51d754-94e3-468b-97cd-d9e6560051e7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.286570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:05.296015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de853aa-abf3-4c62-859e-47dc21f0bdba None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:05.335134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b40c30-3e34-4d20-ae1a-68c47e74d5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:05.335778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b40c30-3e34-4d20-ae1a-68c47e74d5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.336061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b40c30-3e34-4d20-ae1a-68c47e74d5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.336371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b40c30-3e34-4d20-ae1a-68c47e74d5c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.337581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b40c30-3e34-4d20-ae1a-68c47e74d5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:05.339111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b40c30-3e34-4d20-ae1a-68c47e74d5c3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:05.341842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 139/279] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:05.348137 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967848e-42de-48fa-8e95-0a84bf400856 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.348517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967848e-42de-48fa-8e95-0a84bf400856 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.348814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967848e-42de-48fa-8e95-0a84bf400856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.373925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1967848e-42de-48fa-8e95-0a84bf400856 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:05.374952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1967848e-42de-48fa-8e95-0a84bf400856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:05.397300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:05.414223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51d754-94e3-468b-97cd-d9e6560051e7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:05.450479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8f7ae-83c4-4d51-acbc-5b6489724346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:05.451154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f7ae-83c4-4d51-acbc-5b6489724346 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.451435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f7ae-83c4-4d51-acbc-5b6489724346 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.451771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8f7ae-83c4-4d51-acbc-5b6489724346 None admin] PATH_INFO: `/v3/users/5b1ed0a901664727b11e8ca221d7798d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.453027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 140/280] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:05.453861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967848e-42de-48fa-8e95-0a84bf400856 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.467546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8f7ae-83c4-4d51-acbc-5b6489724346 None admin] RBAC: Authorizing `identity:get_user(user_id=5b1ed0a901664727b11e8ca221d7798d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:05.469277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8f7ae-83c4-4d51-acbc-5b6489724346 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:05.470953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 141/281] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:05 2022] GET /identity/v3/users/5b1ed0a901664727b11e8ca221d7798d => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:05.471726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8f7ae-83c4-4d51-acbc-5b6489724346 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.558096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:05.569477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967848e-42de-48fa-8e95-0a84bf400856 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:05.587342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8f7ae-83c4-4d51-acbc-5b6489724346 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:05.606504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56989db6-393b-4349-9617-9d252948c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:05.607114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56989db6-393b-4349-9617-9d252948c5d2 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.607398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56989db6-393b-4349-9617-9d252948c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.607677 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56989db6-393b-4349-9617-9d252948c5d2 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.608363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56989db6-393b-4349-9617-9d252948c5d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:05.609826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56989db6-393b-4349-9617-9d252948c5d2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:05.625061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d7010-edb8-488d-9c98-82e2c1cbdd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:05.625686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d7010-edb8-488d-9c98-82e2c1cbdd28 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.626040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d7010-edb8-488d-9c98-82e2c1cbdd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.626310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d7010-edb8-488d-9c98-82e2c1cbdd28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.637070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d7010-edb8-488d-9c98-82e2c1cbdd28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d29d7010-edb8-488d-9c98-82e2c1cbdd28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:05.642184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:05.643308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 142/282] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:05.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d7010-edb8-488d-9c98-82e2c1cbdd28 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.650981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:05.721689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 141/283] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:18:05 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:18:05.722565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56989db6-393b-4349-9617-9d252948c5d2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.777499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d7010-edb8-488d-9c98-82e2c1cbdd28 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:05.820721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:05.821393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.821537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.821809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.822586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] RBAC: Adding query filter params (role.id=b4f974426b2b44d9843cb577d1b7e396, user.id=5b1ed0a901664727b11e8ca221d7798d, scope.system=all) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:05.822900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:05.824485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:05.833763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 143/284] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 13:18:05 2022] GET /identity/v3/role_assignments?user.id=5b1ed0a901664727b11e8ca221d7798d&scope.system=all&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:05.833763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.870119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56989db6-393b-4349-9617-9d252948c5d2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:05.927354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:05.927997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:05.928328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:05.928605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:05.929308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:05.929619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:05.931031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:05.937752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:05.945912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 142/285] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:05.946319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:05.960657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:05.987077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c708b-1377-44e0-bb2c-0d2875be146f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:06.037441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a2d9f-5781-4ea2-a90d-3e963f5973f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:06.038084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2d9f-5781-4ea2-a90d-3e963f5973f0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:06.038374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2d9f-5781-4ea2-a90d-3e963f5973f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:06.038650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2d9f-5781-4ea2-a90d-3e963f5973f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:06.054760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a2d9f-5781-4ea2-a90d-3e963f5973f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d94a2d9f-5781-4ea2-a90d-3e963f5973f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:06.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:06.061138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:06.061929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 144/286] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:06.068133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a2d9f-5781-4ea2-a90d-3e963f5973f0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:06.089484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936f996-3b60-4e4d-9597-bae295f10bea None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:06.142748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b14b21-315b-4799-b3bd-0b619ade7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:06.144029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b14b21-315b-4799-b3bd-0b619ade7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:06.144603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b14b21-315b-4799-b3bd-0b619ade7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:06.145090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b14b21-315b-4799-b3bd-0b619ade7eb5 None admin] PATH_INFO: `/v3/users/ff4691d8ccdf46a6a7842f51877feebf` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:06.148212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b14b21-315b-4799-b3bd-0b619ade7eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=ff4691d8ccdf46a6a7842f51877feebf)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:06.149762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b14b21-315b-4799-b3bd-0b619ade7eb5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:06.152471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 143/287] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:05 2022] GET /identity/v3/users/ff4691d8ccdf46a6a7842f51877feebf => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:06.158112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b14b21-315b-4799-b3bd-0b619ade7eb5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:06.203518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a2d9f-5781-4ea2-a90d-3e963f5973f0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:06.233616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:06.244208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:06.244894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:06.247479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:06.247479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:06.247479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:06.247479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:06.248326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:06.260493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 145/288] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:06.265972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:06.311844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b14b21-315b-4799-b3bd-0b619ade7eb5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:06.314941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:06.358129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203abc06-c3ec-4edb-84fa-6466515f0fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:06.358688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203abc06-c3ec-4edb-84fa-6466515f0fea None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:06.358971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203abc06-c3ec-4edb-84fa-6466515f0fea None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:06.359243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203abc06-c3ec-4edb-84fa-6466515f0fea None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:06.361624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203abc06-c3ec-4edb-84fa-6466515f0fea None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:06.363410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203abc06-c3ec-4edb-84fa-6466515f0fea None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:06.365623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 144/289] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:06 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:06.371080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203abc06-c3ec-4edb-84fa-6466515f0fea None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:06.402473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592c939-95f4-440d-91e5-175357f3f9fd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:06.489618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406ec0e-64fa-4e47-8357-68adb4db33ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:06.490303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406ec0e-64fa-4e47-8357-68adb4db33ce None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:06.490605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406ec0e-64fa-4e47-8357-68adb4db33ce None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:06.490879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406ec0e-64fa-4e47-8357-68adb4db33ce None admin] PATH_INFO: `/v3/users/cbd65a09d6c945449b9fe2179aa94b00` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:06.492089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406ec0e-64fa-4e47-8357-68adb4db33ce None admin] RBAC: Authorizing `identity:get_user(user_id=cbd65a09d6c945449b9fe2179aa94b00)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:06.493717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406ec0e-64fa-4e47-8357-68adb4db33ce None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:06.501036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 146/290] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:06 2022] GET /identity/v3/users/cbd65a09d6c945449b9fe2179aa94b00 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:06.513745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406ec0e-64fa-4e47-8357-68adb4db33ce None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:06.561327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203abc06-c3ec-4edb-84fa-6466515f0fea None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:06.618077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:06.631561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb1da5-d9ab-4114-a840-5f9e8d1efa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:06.632247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb1da5-d9ab-4114-a840-5f9e8d1efa5f None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:06.632526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb1da5-d9ab-4114-a840-5f9e8d1efa5f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:06.632801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb1da5-d9ab-4114-a840-5f9e8d1efa5f None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92/users/ff4691d8ccdf46a6a7842f51877feebf/roles/c78dd2dd41224a73aec3edfe5d103a84` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:06.638357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb1da5-d9ab-4114-a840-5f9e8d1efa5f None admin] RBAC: Authorizing `identity:create_grant(project_id=7f770973effe4795978f1b5d9c097b92, user_id=ff4691d8ccdf46a6a7842f51877feebf, role_id=c78dd2dd41224a73aec3edfe5d103a84)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:06.641359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb1da5-d9ab-4114-a840-5f9e8d1efa5f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:06.684705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406ec0e-64fa-4e47-8357-68adb4db33ce None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:06.728763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e816b-4d49-4ffe-ab8b-4f9e224a4100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:06.729635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e816b-4d49-4ffe-ab8b-4f9e224a4100 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:06.730091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e816b-4d49-4ffe-ab8b-4f9e224a4100 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:06.730499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e816b-4d49-4ffe-ab8b-4f9e224a4100 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:06.758838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 145/291] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:18:06 2022] PUT /identity/v3/projects/7f770973effe4795978f1b5d9c097b92/users/ff4691d8ccdf46a6a7842f51877feebf/roles/c78dd2dd41224a73aec3edfe5d103a84 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:18:06.819519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e816b-4d49-4ffe-ab8b-4f9e224a4100 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:06.821128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e816b-4d49-4ffe-ab8b-4f9e224a4100 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:06.883812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:06.935304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 147/292] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:18:06.940781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb1da5-d9ab-4114-a840-5f9e8d1efa5f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:06.960341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:07.109138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb1da5-d9ab-4114-a840-5f9e8d1efa5f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:07.151038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155db6fd-e209-48a2-88b5-e4b2e6e4f4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:07.151604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db6fd-e209-48a2-88b5-e4b2e6e4f4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:07.151882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db6fd-e209-48a2-88b5-e4b2e6e4f4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:07.152152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155db6fd-e209-48a2-88b5-e4b2e6e4f4a9 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:07.156987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155db6fd-e209-48a2-88b5-e4b2e6e4f4a9 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:07.158472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155db6fd-e209-48a2-88b5-e4b2e6e4f4a9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:07.160875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 146/293] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:06 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:07.167075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e816b-4d49-4ffe-ab8b-4f9e224a4100 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:07.291729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e816b-4d49-4ffe-ab8b-4f9e224a4100 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:07.336906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520db3c-6212-4992-aaf0-a8ff4d346a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:07.337546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520db3c-6212-4992-aaf0-a8ff4d346a37 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:07.337826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520db3c-6212-4992-aaf0-a8ff4d346a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:07.338096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520db3c-6212-4992-aaf0-a8ff4d346a37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:07.338842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520db3c-6212-4992-aaf0-a8ff4d346a37 None admin] RBAC: Adding query filter params (role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=7f770973effe4795978f1b5d9c097b92, user.id=cbd65a09d6c945449b9fe2179aa94b00) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:07.339147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520db3c-6212-4992-aaf0-a8ff4d346a37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:07.340643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520db3c-6212-4992-aaf0-a8ff4d346a37 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:07.347374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 148/294] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:07 2022] GET /identity/v3/role_assignments?user.id=cbd65a09d6c945449b9fe2179aa94b00&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:08.130760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f0540-ca04-40d9-a35b-3786384dfc30 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.131704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f0540-ca04-40d9-a35b-3786384dfc30 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.131987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f0540-ca04-40d9-a35b-3786384dfc30 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.133675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 147/295] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:08.138837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be8916-afdf-487e-9630-b233230f882c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.139290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be8916-afdf-487e-9630-b233230f882c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.139391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be8916-afdf-487e-9630-b233230f882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.159255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b6be8916-afdf-487e-9630-b233230f882c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:08.160766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b6be8916-afdf-487e-9630-b233230f882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:08.179922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:08.239832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 149/296] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:08.292547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6027cc7-5fd1-472a-b661-1b5bc2f378b7 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.292869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6027cc7-5fd1-472a-b661-1b5bc2f378b7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.293104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6027cc7-5fd1-472a-b661-1b5bc2f378b7 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.294605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 148/297] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:08.300321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809df15-aa27-49a1-90b1-264a47208d81 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.301152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809df15-aa27-49a1-90b1-264a47208d81 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.301422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809df15-aa27-49a1-90b1-264a47208d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.321089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8809df15-aa27-49a1-90b1-264a47208d81 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:08.322288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8809df15-aa27-49a1-90b1-264a47208d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:08.361723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a4ea-5c3b-4d2a-90be-00df9372185a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.362010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a4ea-5c3b-4d2a-90be-00df9372185a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.362291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a4ea-5c3b-4d2a-90be-00df9372185a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.365427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 149/298] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:08.371311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbae0e-fa31-4c45-8db2-5a85e1b69029 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.371605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbae0e-fa31-4c45-8db2-5a85e1b69029 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.371883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbae0e-fa31-4c45-8db2-5a85e1b69029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.393859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-93fbae0e-fa31-4c45-8db2-5a85e1b69029 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:08.395421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-93fbae0e-fa31-4c45-8db2-5a85e1b69029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:08.403625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 150/299] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:08.476281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 150/300] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:08.550062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809df15-aa27-49a1-90b1-264a47208d81 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:08.636467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:08.662216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809df15-aa27-49a1-90b1-264a47208d81 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:08.693138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbae0e-fa31-4c45-8db2-5a85e1b69029 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:08.695823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:08.705127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25212b-2b08-453f-93d6-7152cbd2b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:08.706050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25212b-2b08-453f-93d6-7152cbd2b04d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.711008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25212b-2b08-453f-93d6-7152cbd2b04d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.711008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25212b-2b08-453f-93d6-7152cbd2b04d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.724277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25212b-2b08-453f-93d6-7152cbd2b04d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-6b25212b-2b08-453f-93d6-7152cbd2b04d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:08.731029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:08.731928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:08.732769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:08.733977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 151/301] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:08.740964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25212b-2b08-453f-93d6-7152cbd2b04d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:08.836277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbae0e-fa31-4c45-8db2-5a85e1b69029 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:08.868456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25212b-2b08-453f-93d6-7152cbd2b04d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:08.874043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c63087-101f-4602-a2ad-e5739ea313a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:08.874689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63087-101f-4602-a2ad-e5739ea313a2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.874972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63087-101f-4602-a2ad-e5739ea313a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.875243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63087-101f-4602-a2ad-e5739ea313a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.887913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c63087-101f-4602-a2ad-e5739ea313a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e3c63087-101f-4602-a2ad-e5739ea313a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:08.892731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:08.893585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:08.894970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 151/302] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:08.896430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c63087-101f-4602-a2ad-e5739ea313a2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:08.912028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:08.912722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:08.912941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:08.913216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:08.913900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:08.914204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:08.915595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:08.925524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 152/303] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:08.929376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:08.933757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:08.967329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:09.010246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c63087-101f-4602-a2ad-e5739ea313a2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:09.046463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f50b3-a380-47a5-9de5-cded294ee0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:09.047076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f50b3-a380-47a5-9de5-cded294ee0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.047356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f50b3-a380-47a5-9de5-cded294ee0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.047627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f50b3-a380-47a5-9de5-cded294ee0a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.049034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f50b3-a380-47a5-9de5-cded294ee0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:09.050620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f50b3-a380-47a5-9de5-cded294ee0a8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:09.052236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 152/304] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:09.057691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f50b3-a380-47a5-9de5-cded294ee0a8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:09.059309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6b775-047b-4d81-a6bb-23ab39a074e4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:09.098589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:09.099566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.099856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.100136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.100998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:09.101306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:09.102754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:09.117944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 153/305] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:09.117944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:09.185369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f50b3-a380-47a5-9de5-cded294ee0a8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:09.192952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:09.217958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:09.224200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec275-d641-4c13-aacc-af2ea2458d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:09.224841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec275-d641-4c13-aacc-af2ea2458d95 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.225119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec275-d641-4c13-aacc-af2ea2458d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.225389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec275-d641-4c13-aacc-af2ea2458d95 None admin] PATH_INFO: `/v3/users/5b1ed0a901664727b11e8ca221d7798d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.226582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ec275-d641-4c13-aacc-af2ea2458d95 None admin] RBAC: Authorizing `identity:get_user(user_id=5b1ed0a901664727b11e8ca221d7798d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:09.228036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ec275-d641-4c13-aacc-af2ea2458d95 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:09.229694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 153/306] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:09 2022] GET /identity/v3/users/5b1ed0a901664727b11e8ca221d7798d => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:09.233164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec275-d641-4c13-aacc-af2ea2458d95 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:09.233743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad431f1e-9ac9-4e2b-88fc-07f35da17d8a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:09.269996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130636a1-5036-41c4-a151-f0aad08f5c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:09.270674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130636a1-5036-41c4-a151-f0aad08f5c50 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.270955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130636a1-5036-41c4-a151-f0aad08f5c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.271221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130636a1-5036-41c4-a151-f0aad08f5c50 None admin] PATH_INFO: `/v3/groups` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.271842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130636a1-5036-41c4-a151-f0aad08f5c50 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:09.273315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130636a1-5036-41c4-a151-f0aad08f5c50 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:09.342213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec275-d641-4c13-aacc-af2ea2458d95 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:09.358914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 154/307] 10.222.0.73 () {60 vars in 1134 bytes} [Wed Aug 24 13:18:09 2022] POST /identity/v3/groups => generated 229 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:18:09.359767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130636a1-5036-41c4-a151-f0aad08f5c50 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:09.382322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650ea62-2bb6-4d71-a14a-cc611849cd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:09.382987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ea62-2bb6-4d71-a14a-cc611849cd6b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.383272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ea62-2bb6-4d71-a14a-cc611849cd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.383553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650ea62-2bb6-4d71-a14a-cc611849cd6b None admin] PATH_INFO: `/v3/users/314b4871988c4d0cbeb45c791c0eff0c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.400655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650ea62-2bb6-4d71-a14a-cc611849cd6b None admin] RBAC: Authorizing `identity:get_user(user_id=314b4871988c4d0cbeb45c791c0eff0c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:09.402731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650ea62-2bb6-4d71-a14a-cc611849cd6b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:09.404772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 154/308] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:09 2022] GET /identity/v3/users/314b4871988c4d0cbeb45c791c0eff0c => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:09.406505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03f94-d04b-4ce1-aeed-7b1170ba5982 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.406805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03f94-d04b-4ce1-aeed-7b1170ba5982 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.407061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d03f94-d04b-4ce1-aeed-7b1170ba5982 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.408659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 155/309] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:09.412233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03f94-d04b-4ce1-aeed-7b1170ba5982 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:09.492014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:09.515172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130636a1-5036-41c4-a151-f0aad08f5c50 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:09.545046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:09.547015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d03f94-d04b-4ce1-aeed-7b1170ba5982 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:09.556904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270033c-4ce2-44bf-acb0-f14028f8313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:09.557899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270033c-4ce2-44bf-acb0-f14028f8313b None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.557899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270033c-4ce2-44bf-acb0-f14028f8313b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.558105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270033c-4ce2-44bf-acb0-f14028f8313b None admin] PATH_INFO: `/v3/system/users/5b1ed0a901664727b11e8ca221d7798d/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.561097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270033c-4ce2-44bf-acb0-f14028f8313b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5b1ed0a901664727b11e8ca221d7798d, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:09.563073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270033c-4ce2-44bf-acb0-f14028f8313b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:09.584804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a37e1-61ba-4c15-bf69-c416987cc539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:09.585406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a37e1-61ba-4c15-bf69-c416987cc539 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.585680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a37e1-61ba-4c15-bf69-c416987cc539 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.585949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a37e1-61ba-4c15-bf69-c416987cc539 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.586681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a37e1-61ba-4c15-bf69-c416987cc539 None admin] RBAC: Adding query filter params (role.id=c78dd2dd41224a73aec3edfe5d103a84, user.id=314b4871988c4d0cbeb45c791c0eff0c, scope.system=all) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:09.586984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a37e1-61ba-4c15-bf69-c416987cc539 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:09.588540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a37e1-61ba-4c15-bf69-c416987cc539 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:09.594048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 156/310] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 13:18:09 2022] GET /identity/v3/role_assignments?user.id=314b4871988c4d0cbeb45c791c0eff0c&scope.system=all&role.id=c78dd2dd41224a73aec3edfe5d103a84 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:09.595290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1450d15-3abe-4dc9-ac82-d264a491a09d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:09.595573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1450d15-3abe-4dc9-ac82-d264a491a09d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:09.595852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1450d15-3abe-4dc9-ac82-d264a491a09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:09.621661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c1450d15-3abe-4dc9-ac82-d264a491a09d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:09.622185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c1450d15-3abe-4dc9-ac82-d264a491a09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:09.679739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 155/311] 10.222.0.73 () {58 vars in 1334 bytes} [Wed Aug 24 13:18:09 2022] PUT /identity/v3/system/users/5b1ed0a901664727b11e8ca221d7798d/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:09.715996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 157/312] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:09.987472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270033c-4ce2-44bf-acb0-f14028f8313b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:10.105509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270033c-4ce2-44bf-acb0-f14028f8313b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:10.141633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71965b08-ef63-43f7-a052-dd5bc3a093c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:10.142262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965b08-ef63-43f7-a052-dd5bc3a093c4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:10.142540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965b08-ef63-43f7-a052-dd5bc3a093c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:10.142812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965b08-ef63-43f7-a052-dd5bc3a093c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:10.154961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71965b08-ef63-43f7-a052-dd5bc3a093c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-71965b08-ef63-43f7-a052-dd5bc3a093c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:10.160860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:10.162789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 156/313] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:10.167880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1450d15-3abe-4dc9-ac82-d264a491a09d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:10.224857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:10.279326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1450d15-3abe-4dc9-ac82-d264a491a09d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:10.316533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:10.317139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:10.317422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:10.317696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:10.318368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:10.318668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:10.320088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:10.329365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 158/314] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:10.335224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71965b08-ef63-43f7-a052-dd5bc3a093c4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:10.389156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:10.445547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71965b08-ef63-43f7-a052-dd5bc3a093c4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:10.482184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142bd0e-b2ec-44fe-9f82-9e9509e4e5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:10.482824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142bd0e-b2ec-44fe-9f82-9e9509e4e5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:10.483106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142bd0e-b2ec-44fe-9f82-9e9509e4e5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:10.483381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142bd0e-b2ec-44fe-9f82-9e9509e4e5c9 None admin] PATH_INFO: `/v3/users/ff4691d8ccdf46a6a7842f51877feebf` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:10.484815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142bd0e-b2ec-44fe-9f82-9e9509e4e5c9 None admin] RBAC: Authorizing `identity:get_user(user_id=ff4691d8ccdf46a6a7842f51877feebf)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:10.486283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142bd0e-b2ec-44fe-9f82-9e9509e4e5c9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:10.488743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 157/315] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:10 2022] GET /identity/v3/users/ff4691d8ccdf46a6a7842f51877feebf => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:10.494556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:10.614887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deb685-d315-46ae-8ce4-8fc5cfd84e9e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:10.628834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:10.656792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dbed5-5d8f-43c8-8e37-f9430e59d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:10.657323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbed5-5d8f-43c8-8e37-f9430e59d6cb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:10.657667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbed5-5d8f-43c8-8e37-f9430e59d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:10.657944 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbed5-5d8f-43c8-8e37-f9430e59d6cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:10.749081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dbed5-5d8f-43c8-8e37-f9430e59d6cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:10.749081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dbed5-5d8f-43c8-8e37-f9430e59d6cb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:10.872531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 159/316] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:10.873975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142bd0e-b2ec-44fe-9f82-9e9509e4e5c9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:10.988731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142bd0e-b2ec-44fe-9f82-9e9509e4e5c9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:11.007278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:11.026829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8184e-5667-47e7-9df0-8f41cf0d668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:11.027586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8184e-5667-47e7-9df0-8f41cf0d668b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:11.027870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8184e-5667-47e7-9df0-8f41cf0d668b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:11.028211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8184e-5667-47e7-9df0-8f41cf0d668b None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:11.029441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8184e-5667-47e7-9df0-8f41cf0d668b None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:11.031073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8184e-5667-47e7-9df0-8f41cf0d668b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:11.033130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 158/317] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:10 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:11.039821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dbed5-5d8f-43c8-8e37-f9430e59d6cb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:11.055915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:11.197575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dbed5-5d8f-43c8-8e37-f9430e59d6cb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:11.235475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c3a39-5d42-4fee-8620-7c6cfca99b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:11.236907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c3a39-5d42-4fee-8620-7c6cfca99b54 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:11.237403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c3a39-5d42-4fee-8620-7c6cfca99b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:11.238342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c3a39-5d42-4fee-8620-7c6cfca99b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:11.239385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c3a39-5d42-4fee-8620-7c6cfca99b54 None admin] RBAC: Adding query filter params (role.id=c78dd2dd41224a73aec3edfe5d103a84, scope.project.id=7f770973effe4795978f1b5d9c097b92, user.id=ff4691d8ccdf46a6a7842f51877feebf) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:11.239893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c3a39-5d42-4fee-8620-7c6cfca99b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:11.241684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c3a39-5d42-4fee-8620-7c6cfca99b54 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:11.252299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 160/318] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:11 2022] GET /identity/v3/role_assignments?user.id=ff4691d8ccdf46a6a7842f51877feebf&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=c78dd2dd41224a73aec3edfe5d103a84 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:11.488956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5918c-8582-4f43-9b15-ae0a634aa48f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:11.489494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5918c-8582-4f43-9b15-ae0a634aa48f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:11.489494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5918c-8582-4f43-9b15-ae0a634aa48f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:11.490875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 159/319] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:11.496546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555ae79-75b9-4498-ada8-d68295b5245b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:11.496838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555ae79-75b9-4498-ada8-d68295b5245b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:11.497116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555ae79-75b9-4498-ada8-d68295b5245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:11.515779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7555ae79-75b9-4498-ada8-d68295b5245b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:11.516997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7555ae79-75b9-4498-ada8-d68295b5245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:11.591079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 161/320] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:11.669791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aade6-91f1-4c7e-946a-9d9df046f84e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:11.670078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aade6-91f1-4c7e-946a-9d9df046f84e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:11.670531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aade6-91f1-4c7e-946a-9d9df046f84e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:11.671868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 160/321] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:11.677692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb774ab-8ffe-4079-807d-43754b4cb210 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:11.678075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb774ab-8ffe-4079-807d-43754b4cb210 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:11.678474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb774ab-8ffe-4079-807d-43754b4cb210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:11.684098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e74440-6b77-47f7-880a-5c42698a956b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:11.684592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e74440-6b77-47f7-880a-5c42698a956b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:11.684870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e74440-6b77-47f7-880a-5c42698a956b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:11.686181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 161/322] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:11.691387 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10253904-714c-49a6-9eb6-383ad0790c9a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:11.691676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10253904-714c-49a6-9eb6-383ad0790c9a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:11.691949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10253904-714c-49a6-9eb6-383ad0790c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:11.692734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:11.700276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cdb774ab-8ffe-4079-807d-43754b4cb210 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:11.701489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cdb774ab-8ffe-4079-807d-43754b4cb210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:11.711807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-10253904-714c-49a6-9eb6-383ad0790c9a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:11.713022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-10253904-714c-49a6-9eb6-383ad0790c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:11.788824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 162/323] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:11.798209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 162/324] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:11.865820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb774ab-8ffe-4079-807d-43754b4cb210 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:11.978534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb774ab-8ffe-4079-807d-43754b4cb210 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.020333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2142cc3-38f1-4dc1-af5a-64797dc23784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.020986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2142cc3-38f1-4dc1-af5a-64797dc23784 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.021269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2142cc3-38f1-4dc1-af5a-64797dc23784 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.021543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2142cc3-38f1-4dc1-af5a-64797dc23784 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.022683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2142cc3-38f1-4dc1-af5a-64797dc23784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:12.024123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2142cc3-38f1-4dc1-af5a-64797dc23784 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:12.025889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 163/325] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:12.031403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10253904-714c-49a6-9eb6-383ad0790c9a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.071296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2142cc3-38f1-4dc1-af5a-64797dc23784 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.075172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:12.146036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10253904-714c-49a6-9eb6-383ad0790c9a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.160980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:12.181622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2142cc3-38f1-4dc1-af5a-64797dc23784 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.184958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0e728-759c-4713-9c13-6cffc295e1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.185650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0e728-759c-4713-9c13-6cffc295e1da None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.186067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0e728-759c-4713-9c13-6cffc295e1da None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.186406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0e728-759c-4713-9c13-6cffc295e1da None admin] PATH_INFO: `/v3/groups` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.187123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0e728-759c-4713-9c13-6cffc295e1da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:12.188697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0e728-759c-4713-9c13-6cffc295e1da None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:12.215425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba46cc-affc-450b-bf44-2dc1dbc792e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.216051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba46cc-affc-450b-bf44-2dc1dbc792e5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.216414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba46cc-affc-450b-bf44-2dc1dbc792e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.216699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba46cc-affc-450b-bf44-2dc1dbc792e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.227376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba46cc-affc-450b-bf44-2dc1dbc792e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-84ba46cc-affc-450b-bf44-2dc1dbc792e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:12.232644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:12.233719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:12.234742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 164/326] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:12.237004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba46cc-affc-450b-bf44-2dc1dbc792e5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.277087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 163/327] 10.222.0.73 () {60 vars in 1134 bytes} [Wed Aug 24 13:18:12 2022] POST /identity/v3/groups => generated 226 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:18:12.286054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0e728-759c-4713-9c13-6cffc295e1da None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.308291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:12.357630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba46cc-affc-450b-bf44-2dc1dbc792e5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.403833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeade2e7-ded4-474c-bbaf-42566820db66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.404543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade2e7-ded4-474c-bbaf-42566820db66 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.404836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade2e7-ded4-474c-bbaf-42566820db66 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.405101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeade2e7-ded4-474c-bbaf-42566820db66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.418109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeade2e7-ded4-474c-bbaf-42566820db66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:12.419330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0e728-759c-4713-9c13-6cffc295e1da None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-aeade2e7-ded4-474c-bbaf-42566820db66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:12.423689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:12.424635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:12.425476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 165/328] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:12.434158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeade2e7-ded4-474c-bbaf-42566820db66 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.469251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.469866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.470147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.470421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.471098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:12.471399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:12.473431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:12.484033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 164/329] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:12.490712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.533859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:12.586796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeade2e7-ded4-474c-bbaf-42566820db66 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.618338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:12.623309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62d2cc4-4727-4803-ace3-6d4ed49a5294 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.633580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.634216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.634487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.635127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.635831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:12.636139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:12.637999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:12.649619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 166/330] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:12.657059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.669409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e79cb-4de9-410d-8d94-f8f5166ec596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.670054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e79cb-4de9-410d-8d94-f8f5166ec596 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.670336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e79cb-4de9-410d-8d94-f8f5166ec596 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.670609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e79cb-4de9-410d-8d94-f8f5166ec596 None admin] PATH_INFO: `/v3/users/314b4871988c4d0cbeb45c791c0eff0c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.671898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e79cb-4de9-410d-8d94-f8f5166ec596 None admin] RBAC: Authorizing `identity:get_user(user_id=314b4871988c4d0cbeb45c791c0eff0c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:12.673702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e79cb-4de9-410d-8d94-f8f5166ec596 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:12.676007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 165/331] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:12 2022] GET /identity/v3/users/314b4871988c4d0cbeb45c791c0eff0c => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:12.681306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e79cb-4de9-410d-8d94-f8f5166ec596 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.811301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b599841-b481-4548-82b2-f075a61ddffd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.843770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e79cb-4de9-410d-8d94-f8f5166ec596 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:12.847593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:12.861532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb3b99-3cb4-4ea6-8123-4b51cee7f534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.862760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3b99-3cb4-4ea6-8123-4b51cee7f534 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.863790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3b99-3cb4-4ea6-8123-4b51cee7f534 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.864781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3b99-3cb4-4ea6-8123-4b51cee7f534 None admin] PATH_INFO: `/v3/users/5b1ed0a901664727b11e8ca221d7798d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.866532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb3b99-3cb4-4ea6-8123-4b51cee7f534 None admin] RBAC: Authorizing `identity:get_user(user_id=5b1ed0a901664727b11e8ca221d7798d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:12.868532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb3b99-3cb4-4ea6-8123-4b51cee7f534 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:12.871274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 167/332] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:12 2022] GET /identity/v3/users/5b1ed0a901664727b11e8ca221d7798d => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:12.881316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb3b99-3cb4-4ea6-8123-4b51cee7f534 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:12.897858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6fd22-c2ed-4167-a1e2-395d06e7a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:12.899493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6fd22-c2ed-4167-a1e2-395d06e7a01f None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:12.899934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6fd22-c2ed-4167-a1e2-395d06e7a01f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:12.900788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6fd22-c2ed-4167-a1e2-395d06e7a01f None admin] PATH_INFO: `/v3/system/users/314b4871988c4d0cbeb45c791c0eff0c/roles/c78dd2dd41224a73aec3edfe5d103a84` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:12.902951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6fd22-c2ed-4167-a1e2-395d06e7a01f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=314b4871988c4d0cbeb45c791c0eff0c, role_id=c78dd2dd41224a73aec3edfe5d103a84)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:12.905564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6fd22-c2ed-4167-a1e2-395d06e7a01f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:12.963457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:12.996717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 166/333] 10.222.0.73 () {58 vars in 1334 bytes} [Wed Aug 24 13:18:12 2022] PUT /identity/v3/system/users/314b4871988c4d0cbeb45c791c0eff0c/roles/c78dd2dd41224a73aec3edfe5d103a84 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:13.019851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb3b99-3cb4-4ea6-8123-4b51cee7f534 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:13.067647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685d1d9-972c-465f-941b-9cb6922da25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:13.068319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685d1d9-972c-465f-941b-9cb6922da25f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:13.068611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685d1d9-972c-465f-941b-9cb6922da25f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:13.068895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685d1d9-972c-465f-941b-9cb6922da25f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:13.069692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685d1d9-972c-465f-941b-9cb6922da25f None admin] RBAC: Adding query filter params (role.id=b4f974426b2b44d9843cb577d1b7e396, user.id=5b1ed0a901664727b11e8ca221d7798d, scope.system=all) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:13.070010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685d1d9-972c-465f-941b-9cb6922da25f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:13.071912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685d1d9-972c-465f-941b-9cb6922da25f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:13.087239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 168/334] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 13:18:12 2022] GET /identity/v3/role_assignments?user.id=5b1ed0a901664727b11e8ca221d7798d&scope.system=all&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:14.920842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ab449-d0b2-41cd-b278-a87e5dfd6d4d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:14.921419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ab449-d0b2-41cd-b278-a87e5dfd6d4d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:14.921419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ab449-d0b2-41cd-b278-a87e5dfd6d4d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:14.923009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 167/335] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:14.928913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cca58-fe48-46ae-bd4d-cb463d60796f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:14.929384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cca58-fe48-46ae-bd4d-cb463d60796f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:14.929484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cca58-fe48-46ae-bd4d-cb463d60796f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:14.957320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e28cca58-fe48-46ae-bd4d-cb463d60796f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:14.958774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e28cca58-fe48-46ae-bd4d-cb463d60796f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:15.052887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 169/336] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:15.201095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb093b-5c13-4d3f-a55a-a54b47720d31 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:15.201437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb093b-5c13-4d3f-a55a-a54b47720d31 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:15.201647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb093b-5c13-4d3f-a55a-a54b47720d31 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:15.202995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 168/337] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:15.208813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b597d-6471-4bef-9904-afd033b35fb2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:15.209106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b597d-6471-4bef-9904-afd033b35fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:15.209364 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b597d-6471-4bef-9904-afd033b35fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:15.229930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7f7b597d-6471-4bef-9904-afd033b35fb2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:15.231125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7f7b597d-6471-4bef-9904-afd033b35fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:15.275507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:15.313799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 170/338] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:15.392218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffa98b-b899-41dd-b892-63029d037ab1 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:15.392584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffa98b-b899-41dd-b892-63029d037ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:15.392789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffa98b-b899-41dd-b892-63029d037ab1 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:15.395011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 169/339] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:15.401205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58524-c1ff-4828-9955-1bc455f7a939 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:15.401517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58524-c1ff-4828-9955-1bc455f7a939 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:15.401760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd58524-c1ff-4828-9955-1bc455f7a939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:15.422433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8bd58524-c1ff-4828-9955-1bc455f7a939 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:15.424075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8bd58524-c1ff-4828-9955-1bc455f7a939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:15.494051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffa98b-b899-41dd-b892-63029d037ab1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:15.512503 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 171/340] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:15.661141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffa98b-b899-41dd-b892-63029d037ab1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:15.697890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf67b91-5862-47e7-8c12-eb6d9f826e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:15.698529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf67b91-5862-47e7-8c12-eb6d9f826e24 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:15.698811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf67b91-5862-47e7-8c12-eb6d9f826e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:15.699083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf67b91-5862-47e7-8c12-eb6d9f826e24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:15.711534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf67b91-5862-47e7-8c12-eb6d9f826e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-9cf67b91-5862-47e7-8c12-eb6d9f826e24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:15.717710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:15.718635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:15.719466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 170/341] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:15.728155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58524-c1ff-4828-9955-1bc455f7a939 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:15.753793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf67b91-5862-47e7-8c12-eb6d9f826e24 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:15.791961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:15.887652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf67b91-5862-47e7-8c12-eb6d9f826e24 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:15.929837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8912aa-f742-429b-9c04-60a6539afe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:15.930513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8912aa-f742-429b-9c04-60a6539afe02 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:15.930833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8912aa-f742-429b-9c04-60a6539afe02 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:15.931174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8912aa-f742-429b-9c04-60a6539afe02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:15.946082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8912aa-f742-429b-9c04-60a6539afe02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-0d8912aa-f742-429b-9c04-60a6539afe02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:15.952062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:15.952993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:15.953768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:15.958209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd58524-c1ff-4828-9955-1bc455f7a939 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:15.958655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 171/342] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:18:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:15.962046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:15.967319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8912aa-f742-429b-9c04-60a6539afe02 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:16.013446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509bafe-4bcc-4b30-9e1b-94236e351f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:16.014087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509bafe-4bcc-4b30-9e1b-94236e351f2e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.014370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509bafe-4bcc-4b30-9e1b-94236e351f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.014646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509bafe-4bcc-4b30-9e1b-94236e351f2e None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.015337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509bafe-4bcc-4b30-9e1b-94236e351f2e None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:16.015644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509bafe-4bcc-4b30-9e1b-94236e351f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:16.017265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509bafe-4bcc-4b30-9e1b-94236e351f2e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:16.032320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 172/343] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:16.033757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc8a3a-2b01-43db-8900-bd2a6738d932 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.034093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc8a3a-2b01-43db-8900-bd2a6738d932 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.034401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc8a3a-2b01-43db-8900-bd2a6738d932 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.036264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 173/344] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:16.036828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc8a3a-2b01-43db-8900-bd2a6738d932 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:16.076574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:16.089084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8912aa-f742-429b-9c04-60a6539afe02 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:16.132898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:16.133599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.133883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.134163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.135442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:16.135442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:16.137134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:16.147967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 172/345] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:16.148170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:16.177655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc8a3a-2b01-43db-8900-bd2a6738d932 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:16.221584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87172855-1e0e-4418-9ef3-ef185d601739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:16.222949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87172855-1e0e-4418-9ef3-ef185d601739 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.223228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87172855-1e0e-4418-9ef3-ef185d601739 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.223501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87172855-1e0e-4418-9ef3-ef185d601739 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.238324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87172855-1e0e-4418-9ef3-ef185d601739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-87172855-1e0e-4418-9ef3-ef185d601739 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:16.245636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:16.246585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:16.247484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 174/346] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:16.251958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4358b5-3451-4ef8-a79e-5a6726f8bca1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.251958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4358b5-3451-4ef8-a79e-5a6726f8bca1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.251958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4358b5-3451-4ef8-a79e-5a6726f8bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.268097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa51f3-60cc-441b-a481-de716fa6657f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:16.279087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9f4358b5-3451-4ef8-a79e-5a6726f8bca1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:16.280456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f4358b5-3451-4ef8-a79e-5a6726f8bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:16.301524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:16.339485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed69a8-b007-4f40-a0bb-fbb37827079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:16.340645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed69a8-b007-4f40-a0bb-fbb37827079b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.341152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed69a8-b007-4f40-a0bb-fbb37827079b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.341624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed69a8-b007-4f40-a0bb-fbb37827079b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.366188 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:16.376295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 175/347] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:16.377683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4358b5-3451-4ef8-a79e-5a6726f8bca1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:16.457252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed69a8-b007-4f40-a0bb-fbb37827079b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:16.460628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ed69a8-b007-4f40-a0bb-fbb37827079b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:16.563142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4358b5-3451-4ef8-a79e-5a6726f8bca1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:16.618029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da51df-a04f-4bc3-bb3c-128ba00fafad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:16.618299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da51df-a04f-4bc3-bb3c-128ba00fafad None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.618436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da51df-a04f-4bc3-bb3c-128ba00fafad None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.618848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da51df-a04f-4bc3-bb3c-128ba00fafad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.654361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:16.673470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 173/348] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:16.675406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed69a8-b007-4f40-a0bb-fbb37827079b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:16.708702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da51df-a04f-4bc3-bb3c-128ba00fafad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:16.709949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da51df-a04f-4bc3-bb3c-128ba00fafad None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:16.778606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:16.822833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 176/349] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:16.823796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da51df-a04f-4bc3-bb3c-128ba00fafad None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:16.882910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed69a8-b007-4f40-a0bb-fbb37827079b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:16.885743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:16.936502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da51df-a04f-4bc3-bb3c-128ba00fafad None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:16.953906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:16.956762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.957614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.958361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.959977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:16.961477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:16.965715 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:16.973998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04875e8-9a74-49c0-9a99-95918b47b660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:16.974740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04875e8-9a74-49c0-9a99-95918b47b660 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:16.974870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04875e8-9a74-49c0-9a99-95918b47b660 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:16.975169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04875e8-9a74-49c0-9a99-95918b47b660 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:16.976381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04875e8-9a74-49c0-9a99-95918b47b660 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:16.977892 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04875e8-9a74-49c0-9a99-95918b47b660 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:16.981031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 177/350] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:16 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:16.981915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04875e8-9a74-49c0-9a99-95918b47b660 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:16.987003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 174/351] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:16.989498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:17.093700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04875e8-9a74-49c0-9a99-95918b47b660 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:17.134121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:17.148137 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4b599-1e5d-43cd-9550-24450bc671c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:17.148799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4b599-1e5d-43cd-9550-24450bc671c2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.149078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4b599-1e5d-43cd-9550-24450bc671c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.149412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4b599-1e5d-43cd-9550-24450bc671c2 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.150606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4b599-1e5d-43cd-9550-24450bc671c2 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:17.150714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4863229-788f-40cb-a54d-5b6fba913cde None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:17.152155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4b599-1e5d-43cd-9550-24450bc671c2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:17.157648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 178/352] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:16 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:17.157648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4b599-1e5d-43cd-9550-24450bc671c2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:17.169977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:17.214919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f2d2b-1ac1-45ec-9817-fae6b80ec0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:17.214919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2d2b-1ac1-45ec-9817-fae6b80ec0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.214919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2d2b-1ac1-45ec-9817-fae6b80ec0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.214919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f2d2b-1ac1-45ec-9817-fae6b80ec0f8 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.223342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2d2b-1ac1-45ec-9817-fae6b80ec0f8 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:17.224833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f2d2b-1ac1-45ec-9817-fae6b80ec0f8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:17.226518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 175/353] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:16 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:17.231135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424280b-a627-4b91-a4a5-43790e59a22b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.231414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424280b-a627-4b91-a4a5-43790e59a22b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.232020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424280b-a627-4b91-a4a5-43790e59a22b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.233357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 176/354] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:17.234140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424280b-a627-4b91-a4a5-43790e59a22b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:17.306925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4b599-1e5d-43cd-9550-24450bc671c2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:17.352278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9b971-6182-4040-8aa7-4addf11c7f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:17.352819 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9b971-6182-4040-8aa7-4addf11c7f90 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.353095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9b971-6182-4040-8aa7-4addf11c7f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.353369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9b971-6182-4040-8aa7-4addf11c7f90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.364388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424280b-a627-4b91-a4a5-43790e59a22b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:17.414395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268084-704a-4f2f-9d30-9b6a6b68025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:17.415091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268084-704a-4f2f-9d30-9b6a6b68025e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.415375 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268084-704a-4f2f-9d30-9b6a6b68025e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.415644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa268084-704a-4f2f-9d30-9b6a6b68025e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.430660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa268084-704a-4f2f-9d30-9b6a6b68025e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-fa268084-704a-4f2f-9d30-9b6a6b68025e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:17.437438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:17.438474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 13:18:17.439541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:17.440125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:17.440396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 177/355] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:17.440899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268084-704a-4f2f-9d30-9b6a6b68025e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:17.454197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9b971-6182-4040-8aa7-4addf11c7f90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:17.455669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9b971-6182-4040-8aa7-4addf11c7f90 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:17.487597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:17.569409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa268084-704a-4f2f-9d30-9b6a6b68025e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:17.607867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 179/356] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:17.607867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9b971-6182-4040-8aa7-4addf11c7f90 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:17.620993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea41253-ac4a-4ae2-962f-a8ef8ffaa736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:17.621795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea41253-ac4a-4ae2-962f-a8ef8ffaa736 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.622080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea41253-ac4a-4ae2-962f-a8ef8ffaa736 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.622354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea41253-ac4a-4ae2-962f-a8ef8ffaa736 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.623529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea41253-ac4a-4ae2-962f-a8ef8ffaa736 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:17.625060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea41253-ac4a-4ae2-962f-a8ef8ffaa736 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:17.626797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 178/357] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:17 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:17.631456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294a15-edf3-4bc5-8ba6-ab207baccbf6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.631456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294a15-edf3-4bc5-8ba6-ab207baccbf6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.631456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294a15-edf3-4bc5-8ba6-ab207baccbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.651712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d1294a15-edf3-4bc5-8ba6-ab207baccbf6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:17.653375 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d1294a15-edf3-4bc5-8ba6-ab207baccbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:17.719090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:17.732049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9b971-6182-4040-8aa7-4addf11c7f90 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:17.742364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 179/358] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:17.743017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1294a15-edf3-4bc5-8ba6-ab207baccbf6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:17.775910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6485-dfc6-458f-87da-03fd0a2ee611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:17.776565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6485-dfc6-458f-87da-03fd0a2ee611 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.776851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6485-dfc6-458f-87da-03fd0a2ee611 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.777131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6485-dfc6-458f-87da-03fd0a2ee611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.777863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6485-dfc6-458f-87da-03fd0a2ee611 None admin] RBAC: Adding query filter params (group.id=96c619da054645f6b2cc43009632500e, role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=822c049cabcb4edb8210aa9424fbb06a) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:17.778180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6485-dfc6-458f-87da-03fd0a2ee611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:17.779608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6485-dfc6-458f-87da-03fd0a2ee611 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:17.780690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:17.786875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 180/359] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:17 2022] GET /identity/v3/role_assignments?group.id=96c619da054645f6b2cc43009632500e&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:17.791782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59477f-b04a-4928-9751-0e548179a9ae None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.791782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59477f-b04a-4928-9751-0e548179a9ae None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.791782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59477f-b04a-4928-9751-0e548179a9ae None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.791782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 181/360] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:17.791782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59477f-b04a-4928-9751-0e548179a9ae None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:17.875851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1294a15-edf3-4bc5-8ba6-ab207baccbf6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:17.916874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:17.917653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.918059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.918496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.919315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:17.919794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:17.921337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:17.933538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 180/361] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:17.934409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:17.936615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59477f-b04a-4928-9751-0e548179a9ae None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:17.987147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd48fc5-3512-4eea-aff7-f4a6f4a619c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:17.988003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd48fc5-3512-4eea-aff7-f4a6f4a619c5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.988212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd48fc5-3512-4eea-aff7-f4a6f4a619c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.988544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd48fc5-3512-4eea-aff7-f4a6f4a619c5 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:17.989977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd48fc5-3512-4eea-aff7-f4a6f4a619c5 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:17.991529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd48fc5-3512-4eea-aff7-f4a6f4a619c5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:17.995606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 182/362] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:17 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:17.996170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a224a-5533-4109-a34c-2ab644525e85 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:17.996498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a224a-5533-4109-a34c-2ab644525e85 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:17.996776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a224a-5533-4109-a34c-2ab644525e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:18.019780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ff6a224a-5533-4109-a34c-2ab644525e85 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:18.021078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ff6a224a-5533-4109-a34c-2ab644525e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:18.050269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:18.081470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:18.097619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0051a87-6b05-4e81-a7e9-df29ba91c1ac None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:18.121544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 183/363] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:18.122724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a224a-5533-4109-a34c-2ab644525e85 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:18.149318 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:18.149944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:18.150234 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:18.150509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:18.151381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] RBAC: Adding query filter params (group.id=96c619da054645f6b2cc43009632500e, role.id=47878794de344232a226a95ce5771504, scope.project.id=822c049cabcb4edb8210aa9424fbb06a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:18.151706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:18.153202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:18.162003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 181/364] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:17 2022] GET /identity/v3/role_assignments?group.id=96c619da054645f6b2cc43009632500e&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=47878794de344232a226a95ce5771504 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:18.168211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:18.293881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393620a-7ec3-456a-9b75-8c2eed62b58d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:18.319517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a224a-5533-4109-a34c-2ab644525e85 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:18.339333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a855c-1a2d-4b2f-975f-567a088a3338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:18.339988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a855c-1a2d-4b2f-975f-567a088a3338 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:18.340286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a855c-1a2d-4b2f-975f-567a088a3338 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:18.340562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a855c-1a2d-4b2f-975f-567a088a3338 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:18.341780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a855c-1a2d-4b2f-975f-567a088a3338 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:18.343303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a855c-1a2d-4b2f-975f-567a088a3338 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:18.344959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 182/365] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:18 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:18.345987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a855c-1a2d-4b2f-975f-567a088a3338 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:18.380614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:18.389590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ca20a-97c4-4b74-9afa-12dbb4ec50ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:18.396797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ca20a-97c4-4b74-9afa-12dbb4ec50ec None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:18.396797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ca20a-97c4-4b74-9afa-12dbb4ec50ec None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:18.396797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ca20a-97c4-4b74-9afa-12dbb4ec50ec None admin] PATH_INFO: `/v3/users/314b4871988c4d0cbeb45c791c0eff0c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:18.396797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ca20a-97c4-4b74-9afa-12dbb4ec50ec None admin] RBAC: Authorizing `identity:get_user(user_id=314b4871988c4d0cbeb45c791c0eff0c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:18.397644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ca20a-97c4-4b74-9afa-12dbb4ec50ec None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:18.399688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 184/366] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:18:18 2022] GET /identity/v3/users/314b4871988c4d0cbeb45c791c0eff0c => generated 293 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:18.400502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ca20a-97c4-4b74-9afa-12dbb4ec50ec None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:18.442601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:18.464315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a855c-1a2d-4b2f-975f-567a088a3338 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:18.516903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818177d-5ce8-4c73-bdc5-860250ebbec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:18.517568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818177d-5ce8-4c73-bdc5-860250ebbec0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:18.517854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818177d-5ce8-4c73-bdc5-860250ebbec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:18.518126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818177d-5ce8-4c73-bdc5-860250ebbec0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:18.533423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818177d-5ce8-4c73-bdc5-860250ebbec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5818177d-5ce8-4c73-bdc5-860250ebbec0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:18.539554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:18.540558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:18.541381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:18.542010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 183/367] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:18:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:18.543031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818177d-5ce8-4c73-bdc5-860250ebbec0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:18.576664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ca20a-97c4-4b74-9afa-12dbb4ec50ec None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:18.623094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:18.624239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:18.624675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:18.625104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:18.625981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] RBAC: Adding query filter params (group.id=96c619da054645f6b2cc43009632500e, role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=7f770973effe4795978f1b5d9c097b92) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:18.626437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:18.628800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:18.636828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 185/368] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:18 2022] GET /identity/v3/role_assignments?group.id=96c619da054645f6b2cc43009632500e&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:18.637907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:18.674414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:18.704350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818177d-5ce8-4c73-bdc5-860250ebbec0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:18.755227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:18.777335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54eb41-8da2-4931-bc26-8b40af4c4bb3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:18.802588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:18.803534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:18.803832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:18.804254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:18.805129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] RBAC: Adding query filter params (role.id=c78dd2dd41224a73aec3edfe5d103a84, user.id=314b4871988c4d0cbeb45c791c0eff0c, scope.system=all) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:18.805553 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:18.807226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:18.831294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fbba4-131a-4958-a03a-35ebff82cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:18.832443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fbba4-131a-4958-a03a-35ebff82cb3b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:18.832899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fbba4-131a-4958-a03a-35ebff82cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:18.833319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fbba4-131a-4958-a03a-35ebff82cb3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:18.835938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 184/369] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 13:18:18 2022] GET /identity/v3/role_assignments?user.id=314b4871988c4d0cbeb45c791c0eff0c&scope.system=all&role.id=c78dd2dd41224a73aec3edfe5d103a84 => generated 510 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:18.837276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:18.848078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fbba4-131a-4958-a03a-35ebff82cb3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-065fbba4-131a-4958-a03a-35ebff82cb3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:18.854306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:18.855303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:18.856117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:18.859696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 186/370] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:18:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:18.869075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fbba4-131a-4958-a03a-35ebff82cb3b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:19.015381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fbba4-131a-4958-a03a-35ebff82cb3b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:19.057448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:19.058444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:19.058917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:19.059262 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:19.060049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:19.060461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:19.061946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:19.069887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b19d3-db8d-471c-b793-b016cd49115d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:19.073298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 187/371] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:19.078668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:19.091037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:19.137933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:19.139173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:19.139806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:19.140977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:19.142058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:19.143156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:19.145480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:19.165952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 185/372] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:19.172573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:19.191383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:19.263314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4faa22-498f-4a97-82cf-5f84c12e2ad0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:19.304453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1c1ad-7f5e-4acc-a259-57dee18d196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:19.305400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1c1ad-7f5e-4acc-a259-57dee18d196b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:19.305597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1c1ad-7f5e-4acc-a259-57dee18d196b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:19.305968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1c1ad-7f5e-4acc-a259-57dee18d196b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:19.372263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94771676-cd9b-4df2-9182-5a17107c2ca6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:19.442011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea7ec8-db5e-4194-8f66-670d13aeb2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:19.444743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea7ec8-db5e-4194-8f66-670d13aeb2fc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:19.446145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea7ec8-db5e-4194-8f66-670d13aeb2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:19.446591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea7ec8-db5e-4194-8f66-670d13aeb2fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:19.453963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1c1ad-7f5e-4acc-a259-57dee18d196b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:19.455860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1c1ad-7f5e-4acc-a259-57dee18d196b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:19.474469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:19.593276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea7ec8-db5e-4194-8f66-670d13aeb2fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:19.595238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea7ec8-db5e-4194-8f66-670d13aeb2fc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:19.615545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:19.661541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 188/373] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:18:19.666859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1c1ad-7f5e-4acc-a259-57dee18d196b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:19.763338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 186/374] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:19.770938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea7ec8-db5e-4194-8f66-670d13aeb2fc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:19.835545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1c1ad-7f5e-4acc-a259-57dee18d196b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:19.844540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:19.883328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea7ec8-db5e-4194-8f66-670d13aeb2fc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:19.883533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2aec7-bfed-45c5-95c9-3673339d5bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:19.884368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2aec7-bfed-45c5-95c9-3673339d5bdf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:19.884902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2aec7-bfed-45c5-95c9-3673339d5bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:19.885344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2aec7-bfed-45c5-95c9-3673339d5bdf None admin] PATH_INFO: `/v3/projects/1e28975466b54f229ea6496a2befdfd6` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:19.886753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2aec7-bfed-45c5-95c9-3673339d5bdf None admin] RBAC: Authorizing `identity:get_project(project_id=1e28975466b54f229ea6496a2befdfd6)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:19.888467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2aec7-bfed-45c5-95c9-3673339d5bdf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:19.891714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 189/375] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:19 2022] GET /identity/v3/projects/1e28975466b54f229ea6496a2befdfd6 => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:19.896604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:19.897844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2aec7-bfed-45c5-95c9-3673339d5bdf None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:19.965430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc819d-f29d-4a65-8f93-956e8f517d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:19.966067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc819d-f29d-4a65-8f93-956e8f517d78 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:19.966344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc819d-f29d-4a65-8f93-956e8f517d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:19.966612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc819d-f29d-4a65-8f93-956e8f517d78 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:19.968486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc819d-f29d-4a65-8f93-956e8f517d78 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:19.969931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc819d-f29d-4a65-8f93-956e8f517d78 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:19.972827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 187/376] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:19 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:19.985754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc819d-f29d-4a65-8f93-956e8f517d78 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:20.137099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2aec7-bfed-45c5-95c9-3673339d5bdf None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:20.185704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc819d-f29d-4a65-8f93-956e8f517d78 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:20.204229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be5e50-378f-4238-9283-018fdf57669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:20.204885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be5e50-378f-4238-9283-018fdf57669c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:20.205164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be5e50-378f-4238-9283-018fdf57669c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:20.205440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be5e50-378f-4238-9283-018fdf57669c None admin] PATH_INFO: `/v3/groups/85f84b26d3d14aafa08840daf3fc5c99` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:20.217727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be5e50-378f-4238-9283-018fdf57669c None admin] RBAC: Authorizing `identity:get_group(group_id=85f84b26d3d14aafa08840daf3fc5c99)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:20.219267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be5e50-378f-4238-9283-018fdf57669c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:20.221395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 190/377] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:19 2022] GET /identity/v3/groups/85f84b26d3d14aafa08840daf3fc5c99 => generated 229 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:20.227913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be5e50-378f-4238-9283-018fdf57669c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:20.264922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111c4d5-a626-4e89-b1fa-81adca5b92d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:20.265575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111c4d5-a626-4e89-b1fa-81adca5b92d3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:20.265853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111c4d5-a626-4e89-b1fa-81adca5b92d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:20.266128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111c4d5-a626-4e89-b1fa-81adca5b92d3 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:20.267356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111c4d5-a626-4e89-b1fa-81adca5b92d3 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:20.268927 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111c4d5-a626-4e89-b1fa-81adca5b92d3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:20.271604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 188/378] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:19 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:20.276389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111c4d5-a626-4e89-b1fa-81adca5b92d3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:20.283219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:20.336023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:20.354252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be5e50-378f-4238-9283-018fdf57669c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:20.426664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111c4d5-a626-4e89-b1fa-81adca5b92d3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:20.431197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53d96-9cdf-48db-b66d-8a8178f223eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:20.431897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53d96-9cdf-48db-b66d-8a8178f223eb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:20.432154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53d96-9cdf-48db-b66d-8a8178f223eb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:20.432449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53d96-9cdf-48db-b66d-8a8178f223eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:20.433213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53d96-9cdf-48db-b66d-8a8178f223eb None admin] RBAC: Adding query filter params (group.id=85f84b26d3d14aafa08840daf3fc5c99, role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=1e28975466b54f229ea6496a2befdfd6) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:20.433520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53d96-9cdf-48db-b66d-8a8178f223eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:20.435013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53d96-9cdf-48db-b66d-8a8178f223eb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:20.449725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 191/379] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:20 2022] GET /identity/v3/role_assignments?group.id=85f84b26d3d14aafa08840daf3fc5c99&scope.project.id=1e28975466b54f229ea6496a2befdfd6&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:20.485251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766df45f-82aa-44d3-acec-c78291aa03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:20.485933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df45f-82aa-44d3-acec-c78291aa03b3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:20.486213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df45f-82aa-44d3-acec-c78291aa03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:20.486479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766df45f-82aa-44d3-acec-c78291aa03b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:20.487210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df45f-82aa-44d3-acec-c78291aa03b3 None admin] RBAC: Adding query filter params (group.id=96c619da054645f6b2cc43009632500e, role.id=47878794de344232a226a95ce5771504, scope.project.id=7f770973effe4795978f1b5d9c097b92) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:20.487912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df45f-82aa-44d3-acec-c78291aa03b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:20.489448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766df45f-82aa-44d3-acec-c78291aa03b3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:20.496220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 189/380] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:20 2022] GET /identity/v3/role_assignments?group.id=96c619da054645f6b2cc43009632500e&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=47878794de344232a226a95ce5771504 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:20.578735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d91a-3289-4c05-acda-783ab3f8c2de None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:20.579082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d91a-3289-4c05-acda-783ab3f8c2de None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:20.579293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d91a-3289-4c05-acda-783ab3f8c2de None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:20.581630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 192/381] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:20.594353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1262d-3ce1-42b2-a87b-126f5f59944f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:20.594670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1262d-3ce1-42b2-a87b-126f5f59944f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:20.594905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1262d-3ce1-42b2-a87b-126f5f59944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:20.626624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c5f1262d-3ce1-42b2-a87b-126f5f59944f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:20.628225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c5f1262d-3ce1-42b2-a87b-126f5f59944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:20.723120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 190/382] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:21.010333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d91a-3289-4c05-acda-783ab3f8c2de None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:21.050775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60528fa-5b2d-4372-b719-00bf87412c12 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.051306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60528fa-5b2d-4372-b719-00bf87412c12 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.051859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60528fa-5b2d-4372-b719-00bf87412c12 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.054563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 191/383] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:21.059760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95815cd-0bd3-46c6-8be7-618dcbc5eff2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.060043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95815cd-0bd3-46c6-8be7-618dcbc5eff2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.060362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95815cd-0bd3-46c6-8be7-618dcbc5eff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.097372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a95815cd-0bd3-46c6-8be7-618dcbc5eff2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:21.098752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a95815cd-0bd3-46c6-8be7-618dcbc5eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:21.137363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d91a-3289-4c05-acda-783ab3f8c2de None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:21.139580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:21.169731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:21.177792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7f056-41e1-49bd-a524-49f06b5d4804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:21.178435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7f056-41e1-49bd-a524-49f06b5d4804 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.178720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7f056-41e1-49bd-a524-49f06b5d4804 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.179102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7f056-41e1-49bd-a524-49f06b5d4804 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.193305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7f056-41e1-49bd-a524-49f06b5d4804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d5f7f056-41e1-49bd-a524-49f06b5d4804 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:21.198945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:21.200706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:21.202368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 193/384] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:21.210946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7f056-41e1-49bd-a524-49f06b5d4804 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:21.289684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 192/385] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:21.324931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7f056-41e1-49bd-a524-49f06b5d4804 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:21.363074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:21.363683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.363969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.364314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.365061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] RBAC: Adding query filter params (name=member) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:21.365422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:21.366818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:21.376233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 194/386] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:21.382259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95815cd-0bd3-46c6-8be7-618dcbc5eff2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:21.492078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95815cd-0bd3-46c6-8be7-618dcbc5eff2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:21.529435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0529bb5-a7d6-4204-aa63-6eeeb6424e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:21.530403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0529bb5-a7d6-4204-aa63-6eeeb6424e92 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.530866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0529bb5-a7d6-4204-aa63-6eeeb6424e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.531335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0529bb5-a7d6-4204-aa63-6eeeb6424e92 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.532774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0529bb5-a7d6-4204-aa63-6eeeb6424e92 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:21.535346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0529bb5-a7d6-4204-aa63-6eeeb6424e92 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:21.537773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 193/387] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:21 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:21.545382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:21.576106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:21.664987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0529bb5-a7d6-4204-aa63-6eeeb6424e92 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:21.687117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ab5b6-1f9d-4bea-a0a4-b0a50ce33921 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:21.737047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74c4e-7c08-4157-be77-e39c6e7a6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:21.737759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74c4e-7c08-4157-be77-e39c6e7a6f59 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.738069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74c4e-7c08-4157-be77-e39c6e7a6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.738318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d74c4e-7c08-4157-be77-e39c6e7a6f59 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.742170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d74c4e-7c08-4157-be77-e39c6e7a6f59 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:21.747214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d74c4e-7c08-4157-be77-e39c6e7a6f59 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:21.748704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:21.749059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 195/388] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:21 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:21.755391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74c4e-7c08-4157-be77-e39c6e7a6f59 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:21.883758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0529bb5-a7d6-4204-aa63-6eeeb6424e92 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:21.902762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:21.912095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d74c4e-7c08-4157-be77-e39c6e7a6f59 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:21.931577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a66d8-4d42-4a02-adb8-8568f28aa14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:21.932457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a66d8-4d42-4a02-adb8-8568f28aa14f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.932745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a66d8-4d42-4a02-adb8-8568f28aa14f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.933017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a66d8-4d42-4a02-adb8-8568f28aa14f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.949881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a66d8-4d42-4a02-adb8-8568f28aa14f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-379a66d8-4d42-4a02-adb8-8568f28aa14f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:21.956777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:21.957702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:21.958472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 194/389] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:18:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:21.959686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218809c-5ef3-4dd3-9c18-53963fc97468 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.959686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218809c-5ef3-4dd3-9c18-53963fc97468 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.959851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218809c-5ef3-4dd3-9c18-53963fc97468 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.961355 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 195/390] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:21.962458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179c592-8f9c-4ac2-aa2c-bd22cb74a63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:21.963153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179c592-8f9c-4ac2-aa2c-bd22cb74a63e None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.963439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179c592-8f9c-4ac2-aa2c-bd22cb74a63e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.963715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179c592-8f9c-4ac2-aa2c-bd22cb74a63e None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a/groups/96c619da054645f6b2cc43009632500e/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.966806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2a5c-298f-457e-b6cf-68906c2dd3c1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:21.967088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2a5c-298f-457e-b6cf-68906c2dd3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:21.967190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179c592-8f9c-4ac2-aa2c-bd22cb74a63e None admin] RBAC: Authorizing `identity:create_grant(project_id=822c049cabcb4edb8210aa9424fbb06a, group_id=96c619da054645f6b2cc43009632500e, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:21.967361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f2a5c-298f-457e-b6cf-68906c2dd3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:21.968653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179c592-8f9c-4ac2-aa2c-bd22cb74a63e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:21.989416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d64f2a5c-298f-457e-b6cf-68906c2dd3c1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:21.990647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d64f2a5c-298f-457e-b6cf-68906c2dd3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:22.056633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 196/391] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 13:18:21 2022] PUT /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a/groups/96c619da054645f6b2cc43009632500e/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:18:22.057579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179c592-8f9c-4ac2-aa2c-bd22cb74a63e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:22.079404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 196/392] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:22.242895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179c592-8f9c-4ac2-aa2c-bd22cb74a63e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:22.292228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:22.292862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:22.293144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:22.293420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:22.294127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:22.294422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:22.295856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:22.306148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 197/393] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:22.312017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f2a5c-298f-457e-b6cf-68906c2dd3c1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:22.352884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:22.443847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f2a5c-298f-457e-b6cf-68906c2dd3c1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:22.497635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e064b5e-f733-452f-92a8-959fbc70c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:22.498679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e064b5e-f733-452f-92a8-959fbc70c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:22.499145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e064b5e-f733-452f-92a8-959fbc70c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:22.499606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e064b5e-f733-452f-92a8-959fbc70c8f2 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:22.505093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e064b5e-f733-452f-92a8-959fbc70c8f2 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:22.506896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e064b5e-f733-452f-92a8-959fbc70c8f2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:22.508958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 197/394] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:22 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:22.515063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:22.540326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:22.689863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e064b5e-f733-452f-92a8-959fbc70c8f2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:22.692806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df9f0bd-5c30-4503-bb47-969560d1e26d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:22.780536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a00f7a-3e7c-436a-9995-0d6927221847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:22.781395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a00f7a-3e7c-436a-9995-0d6927221847 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:22.781783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a00f7a-3e7c-436a-9995-0d6927221847 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:22.782168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a00f7a-3e7c-436a-9995-0d6927221847 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:22.790246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a00f7a-3e7c-436a-9995-0d6927221847 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:22.792037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a00f7a-3e7c-436a-9995-0d6927221847 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:22.794301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 198/395] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:22 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:22.805832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a00f7a-3e7c-436a-9995-0d6927221847 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:22.844339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e064b5e-f733-452f-92a8-959fbc70c8f2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:22.847677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:22.899037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62b053-2436-44fa-af02-b29b9b3679fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:22.899794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b053-2436-44fa-af02-b29b9b3679fd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:22.900146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b053-2436-44fa-af02-b29b9b3679fd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:22.900532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b053-2436-44fa-af02-b29b9b3679fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:22.915155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62b053-2436-44fa-af02-b29b9b3679fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-fd62b053-2436-44fa-af02-b29b9b3679fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:22.921536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:22.922474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:22.923792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 198/396] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:22.931240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62b053-2436-44fa-af02-b29b9b3679fd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:23.047580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:23.054715 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62b053-2436-44fa-af02-b29b9b3679fd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:23.074964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a00f7a-3e7c-436a-9995-0d6927221847 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:23.102159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23c5e3-2753-439f-a5b1-8db8df8778a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:23.103737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23c5e3-2753-439f-a5b1-8db8df8778a7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.104226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23c5e3-2753-439f-a5b1-8db8df8778a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.104849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23c5e3-2753-439f-a5b1-8db8df8778a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.105903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23c5e3-2753-439f-a5b1-8db8df8778a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:23.106475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23c5e3-2753-439f-a5b1-8db8df8778a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:23.108787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23c5e3-2753-439f-a5b1-8db8df8778a7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:23.121048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 199/397] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:23.126166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cbea4-feae-44fb-980f-709667576df4 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.126166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cbea4-feae-44fb-980f-709667576df4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.126166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cbea4-feae-44fb-980f-709667576df4 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.126166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 200/398] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:23.130052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cbea4-feae-44fb-980f-709667576df4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:23.135967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f26fe-80df-4d12-b1a5-e707a490b08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:23.137116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f26fe-80df-4d12-b1a5-e707a490b08f None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.137606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f26fe-80df-4d12-b1a5-e707a490b08f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.138103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f26fe-80df-4d12-b1a5-e707a490b08f None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a/groups/96c619da054645f6b2cc43009632500e/roles/47878794de344232a226a95ce5771504` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.141202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f26fe-80df-4d12-b1a5-e707a490b08f None admin] RBAC: Authorizing `identity:create_grant(project_id=822c049cabcb4edb8210aa9424fbb06a, group_id=96c619da054645f6b2cc43009632500e, role_id=47878794de344232a226a95ce5771504)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:23.142888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f26fe-80df-4d12-b1a5-e707a490b08f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:23.151649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:23.236451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 199/399] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 13:18:22 2022] PUT /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a/groups/96c619da054645f6b2cc43009632500e/roles/47878794de344232a226a95ce5771504 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:18:23.239875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b7818-f017-49e4-bae9-3e3f4e824208 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.240591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b7818-f017-49e4-bae9-3e3f4e824208 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.241084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b7818-f017-49e4-bae9-3e3f4e824208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.264739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cbea4-feae-44fb-980f-709667576df4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:23.269394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-eb1b7818-f017-49e4-bae9-3e3f4e824208 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:23.270635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-eb1b7818-f017-49e4-bae9-3e3f4e824208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:23.320099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0fad6-4df4-4184-8df2-5ef174884a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:23.320823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fad6-4df4-4184-8df2-5ef174884a73 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.321064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fad6-4df4-4184-8df2-5ef174884a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.321337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fad6-4df4-4184-8df2-5ef174884a73 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.323039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0fad6-4df4-4184-8df2-5ef174884a73 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:23.325973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0fad6-4df4-4184-8df2-5ef174884a73 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:23.334276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 201/400] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:23 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:23.334276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7295e9-726a-4f1a-8b16-a5a3acda16fe None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.334276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7295e9-726a-4f1a-8b16-a5a3acda16fe None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.334276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7295e9-726a-4f1a-8b16-a5a3acda16fe None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.334276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 202/401] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:23.337351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7295e9-726a-4f1a-8b16-a5a3acda16fe None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:23.376164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 200/402] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:23.378785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb80a1f-9017-4003-894b-92bd5b4cfce9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.379774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb80a1f-9017-4003-894b-92bd5b4cfce9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.380468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb80a1f-9017-4003-894b-92bd5b4cfce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.404826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9cb80a1f-9017-4003-894b-92bd5b4cfce9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:23.406490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9cb80a1f-9017-4003-894b-92bd5b4cfce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:23.438194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:23.457389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7295e9-726a-4f1a-8b16-a5a3acda16fe None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:23.497854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 201/403] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:23.508972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dc722-bfeb-4ccb-a241-22a8acdcd4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:23.510030 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dc722-bfeb-4ccb-a241-22a8acdcd4fb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.510532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dc722-bfeb-4ccb-a241-22a8acdcd4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.511039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dc722-bfeb-4ccb-a241-22a8acdcd4fb None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.512559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dc722-bfeb-4ccb-a241-22a8acdcd4fb None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:23.514368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dc722-bfeb-4ccb-a241-22a8acdcd4fb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:23.516609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 203/404] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:23 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:23.524157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb80a1f-9017-4003-894b-92bd5b4cfce9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:23.527098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:23.646128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb80a1f-9017-4003-894b-92bd5b4cfce9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:23.688343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daabce4-cc65-4c66-8641-e348163e68ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:23.689418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daabce4-cc65-4c66-8641-e348163e68ae None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.689968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daabce4-cc65-4c66-8641-e348163e68ae None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.692164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dc722-bfeb-4ccb-a241-22a8acdcd4fb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:23.693133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daabce4-cc65-4c66-8641-e348163e68ae None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92/groups/96c619da054645f6b2cc43009632500e/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.695368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daabce4-cc65-4c66-8641-e348163e68ae None admin] RBAC: Authorizing `identity:create_grant(project_id=7f770973effe4795978f1b5d9c097b92, group_id=96c619da054645f6b2cc43009632500e, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:23.698188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daabce4-cc65-4c66-8641-e348163e68ae None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:23.818641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dc722-bfeb-4ccb-a241-22a8acdcd4fb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:23.827710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 202/405] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 13:18:23 2022] PUT /identity/v3/projects/7f770973effe4795978f1b5d9c097b92/groups/96c619da054645f6b2cc43009632500e/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:18:23.840978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daabce4-cc65-4c66-8641-e348163e68ae None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:23.870803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d456-bc2e-45e2-810f-5da133c8f97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:23.871875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d456-bc2e-45e2-810f-5da133c8f97f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:23.872398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d456-bc2e-45e2-810f-5da133c8f97f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:23.873037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254d456-bc2e-45e2-810f-5da133c8f97f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:23.889328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254d456-bc2e-45e2-810f-5da133c8f97f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5254d456-bc2e-45e2-810f-5da133c8f97f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:23.895702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:23.896862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:23.897667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:23.904609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 204/406] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:18:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:23.906901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d456-bc2e-45e2-810f-5da133c8f97f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:23.926885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:23.954788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:24.019016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daabce4-cc65-4c66-8641-e348163e68ae None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:24.064101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cdefc-1e38-4a72-aead-1fafc48e8305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:24.064616 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cdefc-1e38-4a72-aead-1fafc48e8305 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.064895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cdefc-1e38-4a72-aead-1fafc48e8305 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.065234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cdefc-1e38-4a72-aead-1fafc48e8305 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.078145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cdefc-1e38-4a72-aead-1fafc48e8305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1c7cdefc-1e38-4a72-aead-1fafc48e8305 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:24.084433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:24.086218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:24.088656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 203/407] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:18:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:24.091904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cdefc-1e38-4a72-aead-1fafc48e8305 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:24.098842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254d456-bc2e-45e2-810f-5da133c8f97f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:24.141703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:24.142766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.143341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.144445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.145328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:24.145831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:24.147498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:24.157386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 205/408] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:24.163812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:24.247083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:24.272870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ff8f9-e85c-43b2-8494-059ad60d09f3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:24.292802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cdefc-1e38-4a72-aead-1fafc48e8305 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:24.312458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfafa5-edd8-4b7a-b9bf-3795bf4820e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:24.313475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfafa5-edd8-4b7a-b9bf-3795bf4820e3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.313961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfafa5-edd8-4b7a-b9bf-3795bf4820e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.314446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfafa5-edd8-4b7a-b9bf-3795bf4820e3 None admin] PATH_INFO: `/v3/groups/85f84b26d3d14aafa08840daf3fc5c99` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.315985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfafa5-edd8-4b7a-b9bf-3795bf4820e3 None admin] RBAC: Authorizing `identity:get_group(group_id=85f84b26d3d14aafa08840daf3fc5c99)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:24.317752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfafa5-edd8-4b7a-b9bf-3795bf4820e3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:24.319984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 206/409] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:24 2022] GET /identity/v3/groups/85f84b26d3d14aafa08840daf3fc5c99 => generated 229 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:24.325687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfafa5-edd8-4b7a-b9bf-3795bf4820e3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:24.345335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:24.371438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:24.372051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.372356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.372626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.373304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:24.373600 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:24.375070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:24.391888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 204/410] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:24.398642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:24.506905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfafa5-edd8-4b7a-b9bf-3795bf4820e3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:24.554258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca591c-0cc1-431d-a93a-d56d97279123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:24.554907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca591c-0cc1-431d-a93a-d56d97279123 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.555184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca591c-0cc1-431d-a93a-d56d97279123 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.555557 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca591c-0cc1-431d-a93a-d56d97279123 None admin] PATH_INFO: `/v3/projects/1e28975466b54f229ea6496a2befdfd6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.557817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca591c-0cc1-431d-a93a-d56d97279123 None admin] RBAC: Authorizing `identity:get_project(project_id=1e28975466b54f229ea6496a2befdfd6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:24.559691 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca591c-0cc1-431d-a93a-d56d97279123 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:24.562443 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 207/411] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:24 2022] GET /identity/v3/projects/1e28975466b54f229ea6496a2befdfd6 => generated 344 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:24.563726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3b991-a4f9-4025-ab2d-38dd132f2840 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.564009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3b991-a4f9-4025-ab2d-38dd132f2840 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.568416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3b991-a4f9-4025-ab2d-38dd132f2840 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.572121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 208/412] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:24.574672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3b991-a4f9-4025-ab2d-38dd132f2840 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:24.589496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79507f-93d5-41ab-b732-83cd0b4dbec2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:24.594642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:24.637017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf8280-5835-4873-bff2-584004b2bbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:24.637677 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf8280-5835-4873-bff2-584004b2bbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.637964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf8280-5835-4873-bff2-584004b2bbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.638242 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf8280-5835-4873-bff2-584004b2bbd9 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.639481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf8280-5835-4873-bff2-584004b2bbd9 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:24.641622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf8280-5835-4873-bff2-584004b2bbd9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:24.645658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 205/413] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:24 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:24.646985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cd046-529b-4571-a05e-5a491b6bfb6c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.647272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cd046-529b-4571-a05e-5a491b6bfb6c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.647548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762cd046-529b-4571-a05e-5a491b6bfb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.667331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-762cd046-529b-4571-a05e-5a491b6bfb6c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:24.668537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-762cd046-529b-4571-a05e-5a491b6bfb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:24.731751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3b991-a4f9-4025-ab2d-38dd132f2840 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:24.762764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:24.775396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 206/414] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:24.776810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cd046-529b-4571-a05e-5a491b6bfb6c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:24.784386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadc476-b0a2-4084-9f04-e0c133571183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:24.785618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadc476-b0a2-4084-9f04-e0c133571183 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.785985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadc476-b0a2-4084-9f04-e0c133571183 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.786369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadc476-b0a2-4084-9f04-e0c133571183 None admin] PATH_INFO: `/v3/projects/1e28975466b54f229ea6496a2befdfd6/groups/85f84b26d3d14aafa08840daf3fc5c99/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.788669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadc476-b0a2-4084-9f04-e0c133571183 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e28975466b54f229ea6496a2befdfd6, group_id=85f84b26d3d14aafa08840daf3fc5c99, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:24.790342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadc476-b0a2-4084-9f04-e0c133571183 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:24.881517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 209/415] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 13:18:24 2022] PUT /identity/v3/projects/1e28975466b54f229ea6496a2befdfd6/groups/85f84b26d3d14aafa08840daf3fc5c99/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:24.896669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762cd046-529b-4571-a05e-5a491b6bfb6c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:24.938471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483e79d-9bfe-4334-ae48-ead8378d64b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:24.939038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e79d-9bfe-4334-ae48-ead8378d64b9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:24.939314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e79d-9bfe-4334-ae48-ead8378d64b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:24.939583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e79d-9bfe-4334-ae48-ead8378d64b9 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:24.941617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483e79d-9bfe-4334-ae48-ead8378d64b9 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:24.943143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483e79d-9bfe-4334-ae48-ead8378d64b9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:24.945262 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 207/416] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:24 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:24.951790 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadc476-b0a2-4084-9f04-e0c133571183 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:25.081319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:25.123575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadc476-b0a2-4084-9f04-e0c133571183 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:25.147289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483e79d-9bfe-4334-ae48-ead8378d64b9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:25.170846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fc62b-2702-4f8a-b8d8-d58c6a8ed631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:25.171515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fc62b-2702-4f8a-b8d8-d58c6a8ed631 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:25.171791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fc62b-2702-4f8a-b8d8-d58c6a8ed631 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:25.172064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fc62b-2702-4f8a-b8d8-d58c6a8ed631 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92/groups/96c619da054645f6b2cc43009632500e/roles/47878794de344232a226a95ce5771504` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:25.177034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fc62b-2702-4f8a-b8d8-d58c6a8ed631 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f770973effe4795978f1b5d9c097b92, group_id=96c619da054645f6b2cc43009632500e, role_id=47878794de344232a226a95ce5771504)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:25.178519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fc62b-2702-4f8a-b8d8-d58c6a8ed631 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:25.214059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:25.275681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 210/417] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 13:18:24 2022] PUT /identity/v3/projects/7f770973effe4795978f1b5d9c097b92/groups/96c619da054645f6b2cc43009632500e/roles/47878794de344232a226a95ce5771504 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:25.281254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483e79d-9bfe-4334-ae48-ead8378d64b9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:25.326114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c83894-fe1f-428a-8e49-05c5c1aee977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:25.326939 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c83894-fe1f-428a-8e49-05c5c1aee977 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:25.327228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c83894-fe1f-428a-8e49-05c5c1aee977 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:25.327531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c83894-fe1f-428a-8e49-05c5c1aee977 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:25.341617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c83894-fe1f-428a-8e49-05c5c1aee977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-29c83894-fe1f-428a-8e49-05c5c1aee977 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:25.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:25.348461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:25.349267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 208/418] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:25.361453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fc62b-2702-4f8a-b8d8-d58c6a8ed631 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:25.528881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fc62b-2702-4f8a-b8d8-d58c6a8ed631 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:25.565985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:25.577338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6fc0c-a1f9-49d6-9960-2367dafe1fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:25.578206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fc0c-a1f9-49d6-9960-2367dafe1fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:25.580775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fc0c-a1f9-49d6-9960-2367dafe1fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:25.580775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fc0c-a1f9-49d6-9960-2367dafe1fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:25.580775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6fc0c-a1f9-49d6-9960-2367dafe1fb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:25.580775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6fc0c-a1f9-49d6-9960-2367dafe1fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:25.581260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6fc0c-a1f9-49d6-9960-2367dafe1fb8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:25.592161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 211/419] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:25.598606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c83894-fe1f-428a-8e49-05c5c1aee977 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:25.735946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c83894-fe1f-428a-8e49-05c5c1aee977 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:25.759319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:25.782981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5add97a-04f2-43c6-9960-31fc24cff600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:25.783514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5add97a-04f2-43c6-9960-31fc24cff600 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:25.783794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5add97a-04f2-43c6-9960-31fc24cff600 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:25.784257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5add97a-04f2-43c6-9960-31fc24cff600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:25.866806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5add97a-04f2-43c6-9960-31fc24cff600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:25.868320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5add97a-04f2-43c6-9960-31fc24cff600 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:25.980728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 209/420] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:25.986455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5add97a-04f2-43c6-9960-31fc24cff600 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:25.987760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1e532-4337-48fd-9f0f-a2a363d333f4 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:25.988271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1e532-4337-48fd-9f0f-a2a363d333f4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:25.988736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1e532-4337-48fd-9f0f-a2a363d333f4 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:25.992312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 212/421] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:25.997620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a372ef-c967-4a5f-978c-ad65f9b6d95e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:25.997903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a372ef-c967-4a5f-978c-ad65f9b6d95e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:25.998158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a372ef-c967-4a5f-978c-ad65f9b6d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:26.004585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:26.035040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-63a372ef-c967-4a5f-978c-ad65f9b6d95e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:26.036319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-63a372ef-c967-4a5f-978c-ad65f9b6d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:26.123332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5add97a-04f2-43c6-9960-31fc24cff600 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:26.151287 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 213/422] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:26.177328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1a745-4c92-4361-b702-c90679b8cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:26.178021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1a745-4c92-4361-b702-c90679b8cf05 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:26.178302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1a745-4c92-4361-b702-c90679b8cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:26.178574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1a745-4c92-4361-b702-c90679b8cf05 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:26.181032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1a745-4c92-4361-b702-c90679b8cf05 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:26.183343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1a745-4c92-4361-b702-c90679b8cf05 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:26.185536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 210/423] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:25 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:26.191846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a372ef-c967-4a5f-978c-ad65f9b6d95e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:26.329707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a372ef-c967-4a5f-978c-ad65f9b6d95e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:26.367232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:26.380411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d48e1-1aba-4199-be15-4e1de2c4b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:26.381051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d48e1-1aba-4199-be15-4e1de2c4b19e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:26.381328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d48e1-1aba-4199-be15-4e1de2c4b19e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:26.390814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d48e1-1aba-4199-be15-4e1de2c4b19e None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:26.390814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d48e1-1aba-4199-be15-4e1de2c4b19e None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:26.390814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d48e1-1aba-4199-be15-4e1de2c4b19e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:26.392051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 214/424] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:26 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:26.398363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1a745-4c92-4361-b702-c90679b8cf05 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:26.434136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f91b58-92e9-4455-bbc8-96c11dd10fe0 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:26.434446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f91b58-92e9-4455-bbc8-96c11dd10fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:26.434731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f91b58-92e9-4455-bbc8-96c11dd10fe0 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:26.436068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 215/425] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:26.443407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1f9ff-60be-49f6-abc7-929e54c4bd25 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:26.443721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1f9ff-60be-49f6-abc7-929e54c4bd25 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:26.444782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1f9ff-60be-49f6-abc7-929e54c4bd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:26.479714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-12d1f9ff-60be-49f6-abc7-929e54c4bd25 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:26.481206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-12d1f9ff-60be-49f6-abc7-929e54c4bd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:26.564410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:26.585883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 216/426] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:26.626453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1a745-4c92-4361-b702-c90679b8cf05 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:26.669468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1f9ff-60be-49f6-abc7-929e54c4bd25 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:26.693958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:26.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:26.694925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:26.695205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:26.695944 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] RBAC: Adding query filter params (group.id=96c619da054645f6b2cc43009632500e, role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=822c049cabcb4edb8210aa9424fbb06a) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:26.696280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:26.697739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:26.716412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 211/427] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:26 2022] GET /identity/v3/role_assignments?group.id=96c619da054645f6b2cc43009632500e&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 611 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:26.809182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1f9ff-60be-49f6-abc7-929e54c4bd25 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:26.836289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:26.854133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a463-4199-4cc1-85f1-013b1036770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:26.854798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a463-4199-4cc1-85f1-013b1036770a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:26.855076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a463-4199-4cc1-85f1-013b1036770a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:26.855353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55a463-4199-4cc1-85f1-013b1036770a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:26.868587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55a463-4199-4cc1-85f1-013b1036770a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ab55a463-4199-4cc1-85f1-013b1036770a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:26.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:26.874383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:26.875163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:26.876275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 217/428] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:18:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:26.882974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:26.990219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a463-4199-4cc1-85f1-013b1036770a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:27.012531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70826422-df99-43aa-9223-5c3b84a4705a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:27.053198 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:27.058594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.058594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.058594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.058594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:27.058594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:27.058594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:27.066973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 212/429] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:27.074783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:27.079872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:27.153869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55a463-4199-4cc1-85f1-013b1036770a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:27.209786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7307e0-f8e7-4c00-b7a4-b52691d99769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:27.210244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7307e0-f8e7-4c00-b7a4-b52691d99769 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.210986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7307e0-f8e7-4c00-b7a4-b52691d99769 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.211273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7307e0-f8e7-4c00-b7a4-b52691d99769 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.224631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7307e0-f8e7-4c00-b7a4-b52691d99769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-fb7307e0-f8e7-4c00-b7a4-b52691d99769 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:27.232078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:27.234272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 13:18:27.236236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:27.244283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 218/430] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:27.244569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3216e-4881-4c10-83a0-c89bda3c379d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:27.244731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7307e0-f8e7-4c00-b7a4-b52691d99769 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:27.288066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c73c5-705f-41ba-aeca-3bb2ae24d935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:27.288741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c73c5-705f-41ba-aeca-3bb2ae24d935 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.289022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c73c5-705f-41ba-aeca-3bb2ae24d935 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.289297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c73c5-705f-41ba-aeca-3bb2ae24d935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.305910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:27.372561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c73c5-705f-41ba-aeca-3bb2ae24d935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:27.373146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7307e0-f8e7-4c00-b7a4-b52691d99769 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:27.374045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c73c5-705f-41ba-aeca-3bb2ae24d935 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:27.377127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:27.413628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:27.414466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.414888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.415301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.416249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:27.416695 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:27.418328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:27.435573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 219/431] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:18:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:27.437212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:27.498448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 213/432] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:27.499952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547026c-bd5b-4fc6-ac89-aa7a727c7cfe None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.500289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547026c-bd5b-4fc6-ac89-aa7a727c7cfe None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.500570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547026c-bd5b-4fc6-ac89-aa7a727c7cfe None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.502034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 214/433] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:27.504646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547026c-bd5b-4fc6-ac89-aa7a727c7cfe None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:27.586716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e9f9f-2a26-4b82-bd10-c0a8a11d73a6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:27.590046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:27.632515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc35414-bc32-48d5-bba2-cf2c594a0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:27.633858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35414-bc32-48d5-bba2-cf2c594a0551 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.634334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35414-bc32-48d5-bba2-cf2c594a0551 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.634610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35414-bc32-48d5-bba2-cf2c594a0551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.670423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547026c-bd5b-4fc6-ac89-aa7a727c7cfe None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:27.673315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:27.715843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad34c28-9171-46c0-b034-78fa0c066336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:27.716513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34c28-9171-46c0-b034-78fa0c066336 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.716832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34c28-9171-46c0-b034-78fa0c066336 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.717088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34c28-9171-46c0-b034-78fa0c066336 None admin] PATH_INFO: `/v3/projects/822c049cabcb4edb8210aa9424fbb06a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.718244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad34c28-9171-46c0-b034-78fa0c066336 None admin] RBAC: Authorizing `identity:get_project(project_id=822c049cabcb4edb8210aa9424fbb06a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:27.719711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad34c28-9171-46c0-b034-78fa0c066336 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:27.721717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 215/434] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:27 2022] GET /identity/v3/projects/822c049cabcb4edb8210aa9424fbb06a => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:27.722985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0fd5a-7bfb-4863-bf1b-195387cdc8db None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.723194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0fd5a-7bfb-4863-bf1b-195387cdc8db None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.723408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0fd5a-7bfb-4863-bf1b-195387cdc8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.735602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc35414-bc32-48d5-bba2-cf2c594a0551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:27.737166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc35414-bc32-48d5-bba2-cf2c594a0551 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:27.746542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-acf0fd5a-7bfb-4863-bf1b-195387cdc8db None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:27.746542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-acf0fd5a-7bfb-4863-bf1b-195387cdc8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:27.868948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 216/435] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:27.870021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0fd5a-7bfb-4863-bf1b-195387cdc8db None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:27.881678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 220/436] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:18:27.883177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39711be5-4dcc-478d-b22e-483d7c9a5354 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:27.883628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39711be5-4dcc-478d-b22e-483d7c9a5354 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:27.883902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39711be5-4dcc-478d-b22e-483d7c9a5354 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:27.885339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 221/437] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:27.887224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39711be5-4dcc-478d-b22e-483d7c9a5354 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:27.890640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:28.012258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0fd5a-7bfb-4863-bf1b-195387cdc8db None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.013766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:28.014147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39711be5-4dcc-478d-b22e-483d7c9a5354 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.058568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64890bb-ef2f-41d2-98b4-cfb0d8da3607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.059201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64890bb-ef2f-41d2-98b4-cfb0d8da3607 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.059480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64890bb-ef2f-41d2-98b4-cfb0d8da3607 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.059752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64890bb-ef2f-41d2-98b4-cfb0d8da3607 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.061014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64890bb-ef2f-41d2-98b4-cfb0d8da3607 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:28.062185 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c763095-794c-46c5-949e-dd9b3866ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.062459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64890bb-ef2f-41d2-98b4-cfb0d8da3607 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:28.063225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763095-794c-46c5-949e-dd9b3866ad72 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.063687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763095-794c-46c5-949e-dd9b3866ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.064043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 217/438] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:27 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:28.064344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c763095-794c-46c5-949e-dd9b3866ad72 None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.065348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf8cb6-b368-4fba-aeb0-e635c5885e05 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.066269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf8cb6-b368-4fba-aeb0-e635c5885e05 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.066813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf8cb6-b368-4fba-aeb0-e635c5885e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.072907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c763095-794c-46c5-949e-dd9b3866ad72 None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:28.074568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c763095-794c-46c5-949e-dd9b3866ad72 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:28.076741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 222/439] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:27 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:28.077458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c763095-794c-46c5-949e-dd9b3866ad72 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.094225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-99bf8cb6-b368-4fba-aeb0-e635c5885e05 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:28.095431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-99bf8cb6-b368-4fba-aeb0-e635c5885e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:28.169084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:28.177294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 218/440] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:28.178185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf8cb6-b368-4fba-aeb0-e635c5885e05 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.206046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c763095-794c-46c5-949e-dd9b3866ad72 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.244866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.245590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.245942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.246228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.246958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] RBAC: Adding query filter params (group.id=96c619da054645f6b2cc43009632500e, role.id=47878794de344232a226a95ce5771504, scope.project.id=822c049cabcb4edb8210aa9424fbb06a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:28.247267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:28.248741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:28.256162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 223/441] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/role_assignments?group.id=96c619da054645f6b2cc43009632500e&scope.project.id=822c049cabcb4edb8210aa9424fbb06a&role.id=47878794de344232a226a95ce5771504 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:28.262031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.292005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:28.295704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf8cb6-b368-4fba-aeb0-e635c5885e05 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.344477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343c39e-1b65-46de-bd4c-53928ecaae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.345497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343c39e-1b65-46de-bd4c-53928ecaae14 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.345861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343c39e-1b65-46de-bd4c-53928ecaae14 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.346204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343c39e-1b65-46de-bd4c-53928ecaae14 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.348902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343c39e-1b65-46de-bd4c-53928ecaae14 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:28.350785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343c39e-1b65-46de-bd4c-53928ecaae14 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:28.353662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 219/442] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:28.359346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343c39e-1b65-46de-bd4c-53928ecaae14 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.409707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24187b-6a11-42ea-80b6-2d5220328c88 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.452833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452d459-4595-4ba2-bb5a-369483dc3257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.453827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d459-4595-4ba2-bb5a-369483dc3257 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.453827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d459-4595-4ba2-bb5a-369483dc3257 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.454027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d459-4595-4ba2-bb5a-369483dc3257 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.457312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:28.467005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452d459-4595-4ba2-bb5a-369483dc3257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-6452d459-4595-4ba2-bb5a-369483dc3257 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:28.472541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:28.473532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:28.474364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 224/443] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:28.475018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452d459-4595-4ba2-bb5a-369483dc3257 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.494079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343c39e-1b65-46de-bd4c-53928ecaae14 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.538867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.539505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.539745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.540022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.540844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] RBAC: Adding query filter params (group.id=96c619da054645f6b2cc43009632500e, role.id=b4f974426b2b44d9843cb577d1b7e396, scope.project.id=7f770973effe4795978f1b5d9c097b92) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:28.541154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:28.543666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:28.550936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 220/444] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/role_assignments?group.id=96c619da054645f6b2cc43009632500e&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=b4f974426b2b44d9843cb577d1b7e396 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:28.551778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.598554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:28.618174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452d459-4595-4ba2-bb5a-369483dc3257 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.663563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88313b-7c84-4e8e-9ff1-d87259586c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.664240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88313b-7c84-4e8e-9ff1-d87259586c0c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.664528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88313b-7c84-4e8e-9ff1-d87259586c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.664804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88313b-7c84-4e8e-9ff1-d87259586c0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.683827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88313b-7c84-4e8e-9ff1-d87259586c0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ed88313b-7c84-4e8e-9ff1-d87259586c0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:28.692203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:28.693248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:28.694027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 225/445] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:28.700847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88313b-7c84-4e8e-9ff1-d87259586c0c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.702025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd7891-7da6-4977-bcc2-026cd3687d88 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.756570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.757195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.757473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.757740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.758426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:28.758727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:28.760125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:28.771480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 221/446] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:28.777935 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.781537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:28.849230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88313b-7c84-4e8e-9ff1-d87259586c0c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.892556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.893438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.894495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.894913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:28.896721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:28.897189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:28.899669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:28.912436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:28.918623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 226/447] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:28.921313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cf94-e33f-4e8b-be95-c00261bad711 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:28.926086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:28.971628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c21e12-4fab-4e5a-87f3-7e9f48159fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:28.972314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21e12-4fab-4e5a-87f3-7e9f48159fad None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:28.972690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21e12-4fab-4e5a-87f3-7e9f48159fad None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:28.972968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c21e12-4fab-4e5a-87f3-7e9f48159fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.038156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eb94d-5b92-41bb-90e0-93a59147d84a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.045203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c21e12-4fab-4e5a-87f3-7e9f48159fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.046789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c21e12-4fab-4e5a-87f3-7e9f48159fad None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.058139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:29.078389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbe9bf-9942-44b5-aa7c-00abbce77926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.079000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe9bf-9942-44b5-aa7c-00abbce77926 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.079291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe9bf-9942-44b5-aa7c-00abbce77926 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.079563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe9bf-9942-44b5-aa7c-00abbce77926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.147106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:29.152026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbe9bf-9942-44b5-aa7c-00abbce77926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.153477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbe9bf-9942-44b5-aa7c-00abbce77926 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.158061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 222/448] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:18:29.161138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c21e12-4fab-4e5a-87f3-7e9f48159fad None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:29.264144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 227/449] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:18:29.270124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbe9bf-9942-44b5-aa7c-00abbce77926 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:29.273843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c21e12-4fab-4e5a-87f3-7e9f48159fad None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.288014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:29.313588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902239f0-3bee-4873-bdc2-10555963ac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.314225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902239f0-3bee-4873-bdc2-10555963ac87 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.314504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902239f0-3bee-4873-bdc2-10555963ac87 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.314776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902239f0-3bee-4873-bdc2-10555963ac87 None admin] PATH_INFO: `/v3/projects/1e28975466b54f229ea6496a2befdfd6` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.315988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902239f0-3bee-4873-bdc2-10555963ac87 None admin] RBAC: Authorizing `identity:get_project(project_id=1e28975466b54f229ea6496a2befdfd6)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.317561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902239f0-3bee-4873-bdc2-10555963ac87 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.319204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 223/450] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:29 2022] GET /identity/v3/projects/1e28975466b54f229ea6496a2befdfd6 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:29.324887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902239f0-3bee-4873-bdc2-10555963ac87 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:29.379061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbe9bf-9942-44b5-aa7c-00abbce77926 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.389020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:29.415065 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6228c-3337-49cf-b5c3-a18f115da87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.415608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6228c-3337-49cf-b5c3-a18f115da87e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.415886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6228c-3337-49cf-b5c3-a18f115da87e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.416155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6228c-3337-49cf-b5c3-a18f115da87e None admin] PATH_INFO: `/v3/projects/7f770973effe4795978f1b5d9c097b92` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.417315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6228c-3337-49cf-b5c3-a18f115da87e None admin] RBAC: Authorizing `identity:get_project(project_id=7f770973effe4795978f1b5d9c097b92)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.418752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6228c-3337-49cf-b5c3-a18f115da87e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.420678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 228/451] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:18:29 2022] GET /identity/v3/projects/7f770973effe4795978f1b5d9c097b92 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:29.426945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6228c-3337-49cf-b5c3-a18f115da87e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:29.434373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902239f0-3bee-4873-bdc2-10555963ac87 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.469970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45281106-0b6f-4f2f-b003-07a0d1815867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.470656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45281106-0b6f-4f2f-b003-07a0d1815867 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.470936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45281106-0b6f-4f2f-b003-07a0d1815867 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.471204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45281106-0b6f-4f2f-b003-07a0d1815867 None admin] PATH_INFO: `/v3/groups/85f84b26d3d14aafa08840daf3fc5c99` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.473065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45281106-0b6f-4f2f-b003-07a0d1815867 None admin] RBAC: Authorizing `identity:get_group(group_id=85f84b26d3d14aafa08840daf3fc5c99)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.474500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45281106-0b6f-4f2f-b003-07a0d1815867 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.476071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 224/452] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:29 2022] GET /identity/v3/groups/85f84b26d3d14aafa08840daf3fc5c99 => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:29.482997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45281106-0b6f-4f2f-b003-07a0d1815867 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:29.526131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:29.544518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6228c-3337-49cf-b5c3-a18f115da87e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.581406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5def7c0-66a3-4ad9-9005-d2a4e2d579f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.582041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5def7c0-66a3-4ad9-9005-d2a4e2d579f8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.582317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5def7c0-66a3-4ad9-9005-d2a4e2d579f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.582583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5def7c0-66a3-4ad9-9005-d2a4e2d579f8 None admin] PATH_INFO: `/v3/groups/96c619da054645f6b2cc43009632500e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.583733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5def7c0-66a3-4ad9-9005-d2a4e2d579f8 None admin] RBAC: Authorizing `identity:get_group(group_id=96c619da054645f6b2cc43009632500e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.585198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5def7c0-66a3-4ad9-9005-d2a4e2d579f8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.586775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 229/453] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 13:18:29 2022] GET /identity/v3/groups/96c619da054645f6b2cc43009632500e => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:29.591874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45281106-0b6f-4f2f-b003-07a0d1815867 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.592664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5def7c0-66a3-4ad9-9005-d2a4e2d579f8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:29.623278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:29.629776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda0628-56dd-410a-8fb5-953ac613d297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.630381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda0628-56dd-410a-8fb5-953ac613d297 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.630664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda0628-56dd-410a-8fb5-953ac613d297 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.630940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda0628-56dd-410a-8fb5-953ac613d297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.631684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda0628-56dd-410a-8fb5-953ac613d297 None admin] RBAC: Adding query filter params (group.id=85f84b26d3d14aafa08840daf3fc5c99, role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=1e28975466b54f229ea6496a2befdfd6) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:29.631980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda0628-56dd-410a-8fb5-953ac613d297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.633454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda0628-56dd-410a-8fb5-953ac613d297 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.639433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 225/454] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:29 2022] GET /identity/v3/role_assignments?group.id=85f84b26d3d14aafa08840daf3fc5c99&scope.project.id=1e28975466b54f229ea6496a2befdfd6&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:29.702901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5def7c0-66a3-4ad9-9005-d2a4e2d579f8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:29.741667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bafdd-18af-4ad6-a476-d3238d2cc749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:29.742284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bafdd-18af-4ad6-a476-d3238d2cc749 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:29.742567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bafdd-18af-4ad6-a476-d3238d2cc749 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:29.742845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bafdd-18af-4ad6-a476-d3238d2cc749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:29.743589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bafdd-18af-4ad6-a476-d3238d2cc749 None admin] RBAC: Adding query filter params (group.id=96c619da054645f6b2cc43009632500e, role.id=47878794de344232a226a95ce5771504, scope.project.id=7f770973effe4795978f1b5d9c097b92) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:29.743996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bafdd-18af-4ad6-a476-d3238d2cc749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:29.745491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bafdd-18af-4ad6-a476-d3238d2cc749 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:29.752464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 230/455] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 13:18:29 2022] GET /identity/v3/role_assignments?group.id=96c619da054645f6b2cc43009632500e&scope.project.id=7f770973effe4795978f1b5d9c097b92&role.id=47878794de344232a226a95ce5771504 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:32.443609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20284bb-503e-4343-9bab-a2b1ba794b74 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:32.444163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20284bb-503e-4343-9bab-a2b1ba794b74 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:32.444163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20284bb-503e-4343-9bab-a2b1ba794b74 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:32.445832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 226/456] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:32.451689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36999f17-fd41-49f2-a59d-31720085e259 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:32.452636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36999f17-fd41-49f2-a59d-31720085e259 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:32.453111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36999f17-fd41-49f2-a59d-31720085e259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:32.476616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-36999f17-fd41-49f2-a59d-31720085e259 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:32.478150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-36999f17-fd41-49f2-a59d-31720085e259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:32.493295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:32.571012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 231/457] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:32.663592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b6531-2158-4c32-9ee6-b440bc719bac None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:32.664266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b6531-2158-4c32-9ee6-b440bc719bac None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:32.664746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b6531-2158-4c32-9ee6-b440bc719bac None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:32.666562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 227/458] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:32.671792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549f347-33d9-43a4-8df1-6cd8d3a630ac None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:32.672083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549f347-33d9-43a4-8df1-6cd8d3a630ac None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:32.672569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549f347-33d9-43a4-8df1-6cd8d3a630ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:32.700471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7549f347-33d9-43a4-8df1-6cd8d3a630ac None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:32.701624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7549f347-33d9-43a4-8df1-6cd8d3a630ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:32.798448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 232/459] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:32.872761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170ddfd-37a9-45ba-9465-075f6237fef2 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:32.873420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170ddfd-37a9-45ba-9465-075f6237fef2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:32.873899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170ddfd-37a9-45ba-9465-075f6237fef2 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:32.875791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 228/460] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:32.881658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619af0e-c8a2-4760-9f97-b9d1b6671f01 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:32.881963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619af0e-c8a2-4760-9f97-b9d1b6671f01 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:32.882214 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619af0e-c8a2-4760-9f97-b9d1b6671f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:32.902077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c619af0e-c8a2-4760-9f97-b9d1b6671f01 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:32.903316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c619af0e-c8a2-4760-9f97-b9d1b6671f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:32.910286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170ddfd-37a9-45ba-9465-075f6237fef2 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:33.023336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 233/461] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:33.120868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170ddfd-37a9-45ba-9465-075f6237fef2 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:33.124011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619af0e-c8a2-4760-9f97-b9d1b6671f01 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:33.139970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:33.195113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:33.195766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.196049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.196422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.223122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:33.224660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:33.259008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:33.259991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:33.260867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:33.260867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:33.260867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:33.260867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 229/462] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:33.267806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:33.334805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619af0e-c8a2-4760-9f97-b9d1b6671f01 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:33.371557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:33.403367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:33.404309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.404737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.405466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.418974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:33.420899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:33.448103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:33.449311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:33.450479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:33.450479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:33.450479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:33.451115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b266a-bb6a-4bf4-ade2-1929e367a464 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:33.456364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 234/463] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:33.456364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:33.502932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:33.503587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.503867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.504142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.504835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:33.505130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:33.506513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:33.524332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 230/464] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:33.524332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:33.545833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:33.595005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c78761-6dfa-4726-834a-a8b844e861a4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:33.639533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a216db-e814-4cf1-9169-42c5b0502648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:33.640658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a216db-e814-4cf1-9169-42c5b0502648 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.641168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a216db-e814-4cf1-9169-42c5b0502648 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.641669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a216db-e814-4cf1-9169-42c5b0502648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.651765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fc19f-db36-4942-8502-9461db77c9c6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:33.656430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a216db-e814-4cf1-9169-42c5b0502648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:33.657909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a216db-e814-4cf1-9169-42c5b0502648 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:33.663839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-55a216db-e814-4cf1-9169-42c5b0502648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:33.675793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:33.686273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 235/465] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d661ea-9576-450c-b6f6-fa8c2186a8b4 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d661ea-9576-450c-b6f6-fa8c2186a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d661ea-9576-450c-b6f6-fa8c2186a8b4 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 236/466] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:33.687118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d661ea-9576-450c-b6f6-fa8c2186a8b4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:33.703841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:33.704772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.705154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.705650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.706925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:33.707353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:33.709856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:33.723732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 231/467] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:33.724578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:33.825142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d661ea-9576-450c-b6f6-fa8c2186a8b4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:33.838780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:33.861305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aee6cb-3b17-44f5-b699-620f1a75368d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:33.871729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c7340-40eb-49d1-8c27-c63f27257344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:33.872750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c7340-40eb-49d1-8c27-c63f27257344 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.873093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c7340-40eb-49d1-8c27-c63f27257344 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.873366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c7340-40eb-49d1-8c27-c63f27257344 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.874358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c7340-40eb-49d1-8c27-c63f27257344 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:33.875857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c7340-40eb-49d1-8c27-c63f27257344 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:33.902742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b95de-6000-48f4-92c1-95de54d18b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:33.903455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b95de-6000-48f4-92c1-95de54d18b32 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.903799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b95de-6000-48f4-92c1-95de54d18b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.904136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b95de-6000-48f4-92c1-95de54d18b32 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.904875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b95de-6000-48f4-92c1-95de54d18b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:33.905241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b95de-6000-48f4-92c1-95de54d18b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:33.906693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b95de-6000-48f4-92c1-95de54d18b32 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:33.919832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 232/468] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:33.921104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d933e8-07e6-42da-9d6b-4fd31fb451c9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:33.921438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d933e8-07e6-42da-9d6b-4fd31fb451c9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:33.922418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d933e8-07e6-42da-9d6b-4fd31fb451c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:33.942038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e6d933e8-07e6-42da-9d6b-4fd31fb451c9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:33.943281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e6d933e8-07e6-42da-9d6b-4fd31fb451c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:33.966539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:33.988697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 237/469] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:18:33 2022] POST /identity/v3/users => generated 246 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:18:33.990013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c7340-40eb-49d1-8c27-c63f27257344 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:34.058268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 233/470] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:34.059447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d933e8-07e6-42da-9d6b-4fd31fb451c9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:34.156522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c7340-40eb-49d1-8c27-c63f27257344 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:34.180422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d933e8-07e6-42da-9d6b-4fd31fb451c9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:34.199635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b889cd1-6607-4afa-aff1-6f1cdca53626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:34.200016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b889cd1-6607-4afa-aff1-6f1cdca53626 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:34.200326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b889cd1-6607-4afa-aff1-6f1cdca53626 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:34.200601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b889cd1-6607-4afa-aff1-6f1cdca53626 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:34.201272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b889cd1-6607-4afa-aff1-6f1cdca53626 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:34.202696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b889cd1-6607-4afa-aff1-6f1cdca53626 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:34.228030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf296453-420d-4d15-be5f-d5a6cc450c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:34.228927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf296453-420d-4d15-be5f-d5a6cc450c42 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:34.229246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf296453-420d-4d15-be5f-d5a6cc450c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:34.229515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf296453-420d-4d15-be5f-d5a6cc450c42 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:34.230337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf296453-420d-4d15-be5f-d5a6cc450c42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:34.232499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf296453-420d-4d15-be5f-d5a6cc450c42 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:34.321737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 238/471] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:18:33 2022] POST /identity/v3/users => generated 248 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:18:34.323353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad47db0-7f73-4b69-a156-405a2c8f374d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:34.323788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad47db0-7f73-4b69-a156-405a2c8f374d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:34.324239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad47db0-7f73-4b69-a156-405a2c8f374d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:34.325722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 239/472] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:34.336211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51072c83-1da0-4987-8b61-6e0ff846023e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:34.336211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51072c83-1da0-4987-8b61-6e0ff846023e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:34.336211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51072c83-1da0-4987-8b61-6e0ff846023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:34.358293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 234/473] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:18:34 2022] POST /identity/v3/users => generated 248 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:18:34.370088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-51072c83-1da0-4987-8b61-6e0ff846023e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:34.371359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-51072c83-1da0-4987-8b61-6e0ff846023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:34.399228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf296453-420d-4d15-be5f-d5a6cc450c42 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:34.401681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:34.431920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:34.520319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 240/474] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:34.591357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf296453-420d-4d15-be5f-d5a6cc450c42 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:34.634077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:34.634729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:34.635009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:34.635277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:34.647140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:34.648750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:34.666934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:34.667898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:34.668790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:34.668790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:34.668790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:34.668790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 235/475] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:34.675703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51072c83-1da0-4987-8b61-6e0ff846023e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:34.787340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51072c83-1da0-4987-8b61-6e0ff846023e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:34.836707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:34.836707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:34.836707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:34.836707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:34.837495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:34.837495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:34.837495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:34.839534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:34.855664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 241/476] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:34.861500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:34.879912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:34.928577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:35.009494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6c6f5-4c26-46ee-aee0-812a22ce7a4d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:35.030709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b18a-adc8-49c2-aa83-35d730d43701 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:35.076217 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511a91-b424-45e1-8c6a-e0ac01a5eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:35.076833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511a91-b424-45e1-8c6a-e0ac01a5eda1 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:35.077109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511a91-b424-45e1-8c6a-e0ac01a5eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:35.078850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b511a91-b424-45e1-8c6a-e0ac01a5eda1 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:35.079514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b511a91-b424-45e1-8c6a-e0ac01a5eda1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:35.081016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b511a91-b424-45e1-8c6a-e0ac01a5eda1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:35.088063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:35.088707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:35.088985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:35.089332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:35.090007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:35.091569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:35.131893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.sql.core [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 13:18:35.131893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 13:18:35.131893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [parameters: {'id': 'be5ff57c07194f6992547a36bbcd7f77', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 13:18:35.131893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81686) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application conn.query(q) Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result.read() Aug 24 13:18:35.150285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 13:18:35.151240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application next(self.gen) Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application yield resource Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application next(self.gen) Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise self.value Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application session.commit() Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 13:18:35.152280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.session.flush() Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application compat.raise_( Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 13:18:35.153146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 13:18:35.154029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application conn.query(q) Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result.read() Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 13:18:35.154846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [parameters: {'id': 'be5ff57c07194f6992547a36bbcd7f77', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:35.155622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 13:18:35.158025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:35.160477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 236/477] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 13:18:34 2022] POST /identity/v3/roles => generated 144 bytes in 313 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:35.160477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:35.199532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 242/478] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:18:34 2022] POST /identity/v3/users => generated 249 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:18:35.300366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb94e8-bf54-42f1-89f7-cf90acd466c8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:35.336936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:35.341702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c4bdf-ef2c-434c-bb48-dfa05ce13281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:35.342504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4bdf-ef2c-434c-bb48-dfa05ce13281 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:35.342988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4bdf-ef2c-434c-bb48-dfa05ce13281 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:35.343444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4bdf-ef2c-434c-bb48-dfa05ce13281 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:35.357800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c4bdf-ef2c-434c-bb48-dfa05ce13281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-826c4bdf-ef2c-434c-bb48-dfa05ce13281 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:35.363654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:35.364821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 13:18:35.365813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:35.368081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 237/479] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:18:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:35.373283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511a91-b424-45e1-8c6a-e0ac01a5eda1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:35.482309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:35.522340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b511a91-b424-45e1-8c6a-e0ac01a5eda1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:35.559775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c7a6b-c0af-4564-8cc4-381e33295eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:35.560342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c7a6b-c0af-4564-8cc4-381e33295eef None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:35.560626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c7a6b-c0af-4564-8cc4-381e33295eef None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:35.560902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c7a6b-c0af-4564-8cc4-381e33295eef None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:35.561580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c7a6b-c0af-4564-8cc4-381e33295eef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:35.561884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c7a6b-c0af-4564-8cc4-381e33295eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:35.563264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c7a6b-c0af-4564-8cc4-381e33295eef None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:35.573172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 243/480] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:36.263553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b70d28-385f-4d27-8449-88b7e745a2a2 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:36.264099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b70d28-385f-4d27-8449-88b7e745a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:36.264099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b70d28-385f-4d27-8449-88b7e745a2a2 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:36.265757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 238/481] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:36.271923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1c360-ed60-4c71-b290-8a50492df8aa None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:36.272241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1c360-ed60-4c71-b290-8a50492df8aa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:36.272521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1c360-ed60-4c71-b290-8a50492df8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:36.297186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4fa1c360-ed60-4c71-b290-8a50492df8aa None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:36.298659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4fa1c360-ed60-4c71-b290-8a50492df8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:36.399040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 244/482] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:36.879445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b70d28-385f-4d27-8449-88b7e745a2a2 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:36.948353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4879de7-8b65-4d73-9c96-f740705c6871 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:36.948887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4879de7-8b65-4d73-9c96-f740705c6871 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:36.948887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4879de7-8b65-4d73-9c96-f740705c6871 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:36.950763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 245/483] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:36.957315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d1c7-0c46-411e-90f3-3f8cc71ad06c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:36.957671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d1c7-0c46-411e-90f3-3f8cc71ad06c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:36.957869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6d1c7-0c46-411e-90f3-3f8cc71ad06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:36.984742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fcb6d1c7-0c46-411e-90f3-3f8cc71ad06c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:36.987903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fcb6d1c7-0c46-411e-90f3-3f8cc71ad06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:37.055063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b70d28-385f-4d27-8449-88b7e745a2a2 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:37.112048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:37.131017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 246/484] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:37.132433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ef87c-6cd6-446d-a7d8-085e3f439c73 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.132724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ef87c-6cd6-446d-a7d8-085e3f439c73 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.133000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ef87c-6cd6-446d-a7d8-085e3f439c73 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.134586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 247/485] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:37.135068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecc2d9-1f77-4bb6-91e2-7c5a53cb4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:37.135813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecc2d9-1f77-4bb6-91e2-7c5a53cb4db0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.136099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecc2d9-1f77-4bb6-91e2-7c5a53cb4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.136402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecc2d9-1f77-4bb6-91e2-7c5a53cb4db0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.139902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e532a1-256d-4e25-8b03-afaf84add973 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.140225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e532a1-256d-4e25-8b03-afaf84add973 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.140507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e532a1-256d-4e25-8b03-afaf84add973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.169200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-15e532a1-256d-4e25-8b03-afaf84add973 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:37.170435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-15e532a1-256d-4e25-8b03-afaf84add973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:37.173536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecc2d9-1f77-4bb6-91e2-7c5a53cb4db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d1ecc2d9-1f77-4bb6-91e2-7c5a53cb4db0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:37.179245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:37.180121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:37.180962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 239/486] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:37.186482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecc2d9-1f77-4bb6-91e2-7c5a53cb4db0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:37.244596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:37.302112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecc2d9-1f77-4bb6-91e2-7c5a53cb4db0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:37.339927 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 248/487] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:37.340415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:37.341093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.341375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.341652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.342334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:37.342643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:37.352800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:37.362785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 240/488] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:37.368344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e532a1-256d-4e25-8b03-afaf84add973 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:37.438354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:37.527832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e532a1-256d-4e25-8b03-afaf84add973 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:37.584749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:37.610400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:37.611117 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.611402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.611786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.638421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ea3bb-be6a-4147-9f15-83a846340a65 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:37.641165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:37.642663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:37.661690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:37.677675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:37.678663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:37.679499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:37.679499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:37.679499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:37.693392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 249/489] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:37.696309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:37.711159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3883e-e3d6-46ad-ac94-eb5c3b75400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:37.711820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3883e-e3d6-46ad-ac94-eb5c3b75400c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.712102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3883e-e3d6-46ad-ac94-eb5c3b75400c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.712983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3883e-e3d6-46ad-ac94-eb5c3b75400c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.726124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3883e-e3d6-46ad-ac94-eb5c3b75400c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-6be3883e-e3d6-46ad-ac94-eb5c3b75400c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:37.731772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:37.732763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:37.741076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 241/490] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:37.748233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3883e-e3d6-46ad-ac94-eb5c3b75400c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:37.835273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73849350-7cf8-4a92-aa1d-43872192dd7a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:37.873321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:37.874318 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.874736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.875399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.876756 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:37.877495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:37.879872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:37.885748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3883e-e3d6-46ad-ac94-eb5c3b75400c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:37.895679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 250/491] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:37.896736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:37.929662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876cecc-2b80-4e81-9d54-0693fe6d878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:37.930291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876cecc-2b80-4e81-9d54-0693fe6d878a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.930577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876cecc-2b80-4e81-9d54-0693fe6d878a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.930920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876cecc-2b80-4e81-9d54-0693fe6d878a None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.931613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876cecc-2b80-4e81-9d54-0693fe6d878a None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:37.931931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876cecc-2b80-4e81-9d54-0693fe6d878a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:37.933362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876cecc-2b80-4e81-9d54-0693fe6d878a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:37.939354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:37.947273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 242/492] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:37.948678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6bbc-283b-4a09-a965-89f51758cff0 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:37.948976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6bbc-283b-4a09-a965-89f51758cff0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:37.954931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6bbc-283b-4a09-a965-89f51758cff0 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:37.954931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 243/493] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:37.954931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e6bbc-283b-4a09-a965-89f51758cff0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:37.986244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:38.031011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53c4e6-9bd7-49cb-87cc-b03743319e31 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.075292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e6bbc-283b-4a09-a965-89f51758cff0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.077555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc0b6-3c25-46ee-9585-9b15afbceb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.078191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc0b6-3c25-46ee-9585-9b15afbceb21 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.078476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc0b6-3c25-46ee-9585-9b15afbceb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.078751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc0b6-3c25-46ee-9585-9b15afbceb21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.090129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842fc0b6-3c25-46ee-9585-9b15afbceb21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-842fc0b6-3c25-46ee-9585-9b15afbceb21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:38.094845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:38.097700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 251/494] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:38.099905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc0b6-3c25-46ee-9585-9b15afbceb21 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:38.122946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96be4bb-743f-4798-aa1a-3fa68993e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.123679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be4bb-743f-4798-aa1a-3fa68993e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.123858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be4bb-743f-4798-aa1a-3fa68993e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.124131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be4bb-743f-4798-aa1a-3fa68993e8a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.131623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96be4bb-743f-4798-aa1a-3fa68993e8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:38.133125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96be4bb-743f-4798-aa1a-3fa68993e8a5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b96be4bb-743f-4798-aa1a-3fa68993e8a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:38.146770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:38.147717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:38.148846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 244/495] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:18:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:38.149680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79693a-b836-4d8e-97cb-a3a0ec4d420b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.149949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79693a-b836-4d8e-97cb-a3a0ec4d420b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.150225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79693a-b836-4d8e-97cb-a3a0ec4d420b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.168685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-eb79693a-b836-4d8e-97cb-a3a0ec4d420b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:38.169877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-eb79693a-b836-4d8e-97cb-a3a0ec4d420b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:38.197659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:38.206389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc0b6-3c25-46ee-9585-9b15afbceb21 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.239722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 245/496] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:38.241008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f568335-ebb4-47a5-ac42-6dd98ef21087 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.241292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f568335-ebb4-47a5-ac42-6dd98ef21087 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.241563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f568335-ebb4-47a5-ac42-6dd98ef21087 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.242972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 246/497] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:38.243077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.243652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.243985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.244232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.247619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f568335-ebb4-47a5-ac42-6dd98ef21087 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:38.255666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:38.257326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:38.273205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:38.274145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:38.274989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:38.274989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:38.274989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:38.274989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 252/498] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:38.275213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:38.276327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:38.359554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f568335-ebb4-47a5-ac42-6dd98ef21087 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.390233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92d0df-3d44-46f8-abcd-3e37067d6233 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.402804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24510d4-212f-4814-9b8f-ee29c5178c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.403414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24510d4-212f-4814-9b8f-ee29c5178c72 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.403751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24510d4-212f-4814-9b8f-ee29c5178c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.404317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24510d4-212f-4814-9b8f-ee29c5178c72 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.405482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24510d4-212f-4814-9b8f-ee29c5178c72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:38.405789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24510d4-212f-4814-9b8f-ee29c5178c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:38.407355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24510d4-212f-4814-9b8f-ee29c5178c72 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:38.416305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 247/499] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:38.417443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaefda-66c3-4c06-b8f2-5e21aac7673b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.417730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaefda-66c3-4c06-b8f2-5e21aac7673b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.418001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeaefda-66c3-4c06-b8f2-5e21aac7673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.435230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.436795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.437259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.437999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.438952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:38.439483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:38.441823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:38.442114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-abeaefda-66c3-4c06-b8f2-5e21aac7673b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:38.443310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-abeaefda-66c3-4c06-b8f2-5e21aac7673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:38.494239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 253/500] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:18:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:38.494239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:38.494239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:38.520787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 248/501] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:38.528409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeaefda-66c3-4c06-b8f2-5e21aac7673b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:38.539656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:38.604077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880863cf-8c4c-49a4-befd-1036ff46038b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.640678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.641200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.641483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.641764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.641872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeaefda-66c3-4c06-b8f2-5e21aac7673b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.642409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:38.642713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:38.645207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:38.656804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 254/502] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:38.657612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:38.688914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.689550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.689922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.690808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.701437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:38.703045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:38.719135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:38.720054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:38.721023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:38.721023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:38.721023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:38.721023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 249/503] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:38.721826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:38.732972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:38.777647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828569e-a89f-4b00-9e9c-0bb09dd3045a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.797307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:38.817218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ed716-efbc-4f10-9b77-15f424a251ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.817821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed716-efbc-4f10-9b77-15f424a251ae None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.818109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed716-efbc-4f10-9b77-15f424a251ae None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.818379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ed716-efbc-4f10-9b77-15f424a251ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.840985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12e044-df75-4619-9559-4a218be90216 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:38.877372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be76b89-038c-41f3-83f4-1ac80711c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:38.878109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76b89-038c-41f3-83f4-1ac80711c84e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:38.878816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76b89-038c-41f3-83f4-1ac80711c84e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:38.879092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76b89-038c-41f3-83f4-1ac80711c84e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:38.889374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be76b89-038c-41f3-83f4-1ac80711c84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:38.894031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ed716-efbc-4f10-9b77-15f424a251ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1be76b89-038c-41f3-83f4-1ac80711c84e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:38.894750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:38.895670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:38.896868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:38.897817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ed716-efbc-4f10-9b77-15f424a251ae None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:38.897887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 250/504] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:38.897887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be76b89-038c-41f3-83f4-1ac80711c84e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:38.957251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.001289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be76b89-038c-41f3-83f4-1ac80711c84e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.001528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 255/505] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:39.002325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ed716-efbc-4f10-9b77-15f424a251ae None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.027955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.037312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.037942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.040109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.040109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.045973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.047431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:39.059228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:39.060166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 251/506] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:39.061115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.107992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ed716-efbc-4f10-9b77-15f424a251ae None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.142483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.143307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.143592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.143866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.144591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:39.144849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.146224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.157239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 256/507] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:39.158024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.168686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc97a2-8a13-4b93-ac72-6dcb2a301779 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.185853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.206031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.206651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.206930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.207197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.217741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.219215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:39.233593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:39.234508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:39.235352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:39.235352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:39.235352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:39.235352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 252/508] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:39.235569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.262966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59c66d-f43c-482a-bec1-26c64b5f7ee7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.276877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.297177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.297780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.298326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.298601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.300307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:39.300622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.302006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.310329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 257/509] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:39.311133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.337498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a986488-d2ad-4ea1-a280-af9bdaa96df0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.371227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.371884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.372117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.372419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.383660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.385143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:39.399506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:39.400644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:39.401667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:39.401667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:39.401667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:39.401868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.401950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 253/510] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:39.401950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.419932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9b969-8e4a-45ea-91ba-001b6bf5c2bc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.454837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.455454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.455730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.456004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.456755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:39.457063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.458522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.476429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 258/511] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:39.477412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.495656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.506209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d7f40-8319-4008-8d4d-224687d16e26 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.541622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.542245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.542522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.542795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.550697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.552130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:39.563016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:39.564105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 13:18:39.566577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:39.568321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 254/512] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:39.569439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.585331 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad6556-1eae-41fa-afdb-200c8b5cb175 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.621833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.622430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.622711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.622993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.623635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:39.623933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.625326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.636464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 259/513] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:39.637251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.639935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.677137 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2175f-3030-44ab-9054-9b905fac7eb5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.710868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.711473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.711747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.712015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.722988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.724465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:39.738995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:39.740083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:39.745747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:39.745747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:39.745747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:39.745747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 255/514] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:39.745747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.746017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2573-617f-4b07-bb4f-50bf2077c302 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.746098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.777351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.777943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.778221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.778496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.779143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:39.779439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.780855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.793134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 260/515] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:39.794015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.853607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20dc7f-b8a6-4127-83ad-91e73723520b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.864984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:39.888532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78e232-def5-45f7-b9a7-571d525fa648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.889142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78e232-def5-45f7-b9a7-571d525fa648 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.889427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78e232-def5-45f7-b9a7-571d525fa648 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.889698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78e232-def5-45f7-b9a7-571d525fa648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.905691 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b5102-2237-4280-922f-eed16a1d6ad0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:39.940948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:39.941458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:39.941744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:39.942008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:39.942722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:39.943025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.944478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.961105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78e232-def5-45f7-b9a7-571d525fa648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:39.961707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 261/516] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:39.962480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:39.962581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78e232-def5-45f7-b9a7-571d525fa648 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:39.965012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:40.062685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5a394-1f66-4ea5-ab20-d68aa2ae2946 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:40.064425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 256/517] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:40.065941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78e232-def5-45f7-b9a7-571d525fa648 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:40.085125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:40.099384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f47b4-a0e2-4982-ba22-bb001e7b6e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:40.099979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f47b4-a0e2-4982-ba22-bb001e7b6e5f None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:40.100285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f47b4-a0e2-4982-ba22-bb001e7b6e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:40.100554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f47b4-a0e2-4982-ba22-bb001e7b6e5f None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:40.101198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f47b4-a0e2-4982-ba22-bb001e7b6e5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:40.102729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f47b4-a0e2-4982-ba22-bb001e7b6e5f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:40.169076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78e232-def5-45f7-b9a7-571d525fa648 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:40.170745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:40.204990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:40.205559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:40.205837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:40.206107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:40.206752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:40.207048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:40.208449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:40.221562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 257/518] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:40.222449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:40.233335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 262/519] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:18:39 2022] POST /identity/v3/users => generated 247 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:18:40.239300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f47b4-a0e2-4982-ba22-bb001e7b6e5f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:40.369697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f47b4-a0e2-4982-ba22-bb001e7b6e5f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:40.373271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b8909-43be-4185-b974-12764a6bf27f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:40.419443 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a33bd6-b777-4c71-9e80-e98b0b6b4332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:40.420857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33bd6-b777-4c71-9e80-e98b0b6b4332 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:40.420857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33bd6-b777-4c71-9e80-e98b0b6b4332 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:40.420857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33bd6-b777-4c71-9e80-e98b0b6b4332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:40.422788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:40.424862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:40.426182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:40.426550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:40.436936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:40.438680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:40.453409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:40.454299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 258/520] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:40.456231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:40.485803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:40.488513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:40.511338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a33bd6-b777-4c71-9e80-e98b0b6b4332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:40.512850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a33bd6-b777-4c71-9e80-e98b0b6b4332 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:40.584236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c27001-ff63-4831-aaf2-45f3d49cbeb8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:40.632991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:40.634924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:40.634924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:40.634924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:40.641138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 263/521] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:40.641138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a33bd6-b777-4c71-9e80-e98b0b6b4332 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:40.648493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:40.649996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:40.665391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:40.666491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:40.667327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:40.667327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:40.667327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:40.667327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 259/522] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:40.667327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:40.742617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:40.774300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a33bd6-b777-4c71-9e80-e98b0b6b4332 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:40.781073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:40.810896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eead6d-c400-4124-adb1-d6fb34bbf156 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:40.816469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:40.817117 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:40.817401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:40.817677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:40.828756 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:40.832442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:40.860344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:40.861503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 13:18:40.863365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:40.864219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 264/523] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:40.865762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:40.880438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:40.881088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:40.881370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:40.881640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:40.882346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:40.882700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:40.886489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:40.921315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 260/524] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:40.922787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:41.012135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173476c-ba34-4253-9516-4e21415cb878 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:41.058980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55222c9-250f-41e0-8409-99acc8edb109 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:41.075642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:41.075642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:41.075642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:41.075642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:41.079118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:41.092429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:41.093950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:41.112228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:41.112815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:41.113096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:41.113373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:41.113519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:41.114440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:41.115289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:41.115289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:41.115289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:41.115462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:41.115462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:41.115590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 265/525] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:41.116083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:41.120469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:41.122351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:41.134731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 261/526] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:41.140071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:41.256238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f43e6-d92a-416b-a688-cbd240b6c8d5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:41.275453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81c431-bbc2-44f8-8628-f229762d090e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:41.305625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:41.306366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:41.306816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:41.307273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:41.308634 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:41.309003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:41.310516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:41.328341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:41.328955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:41.329233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:41.329502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:41.330294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=40561a306123461d92bc669be3962db3) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:41.330747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:41.333363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:41.351248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 266/527] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 13:18:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:41.351248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:41.351553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 262/528] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:41 2022] GET /identity/v3/role_assignments?user.id=40561a306123461d92bc669be3962db3&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:41.351553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:41.432170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:41.492841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:41.523929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9ddb4-62ae-4229-9180-9c2cfee05fae None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:41.561183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f50813-17df-4779-a84f-269d28579290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:41.562139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50813-17df-4779-a84f-269d28579290 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:41.562139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50813-17df-4779-a84f-269d28579290 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:41.562400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50813-17df-4779-a84f-269d28579290 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:41.562934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50813-17df-4779-a84f-269d28579290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:41.563243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50813-17df-4779-a84f-269d28579290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:41.564917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50813-17df-4779-a84f-269d28579290 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:41.567011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6c998-4f20-4c04-952a-097964ac3710 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:41.583603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 267/529] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:41.583603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f50813-17df-4779-a84f-269d28579290 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:41.633516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:41.635536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:41.636108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:41.636716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:41.653659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:41.657469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:41.684235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:41.685758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:41.687191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 263/530] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:41.688036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:41.779053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f50813-17df-4779-a84f-269d28579290 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:41.790252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:41.830067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64669a-ea95-445d-8de2-0689b91924b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:41.832956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64669a-ea95-445d-8de2-0689b91924b3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:41.833733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64669a-ea95-445d-8de2-0689b91924b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:41.834292 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64669a-ea95-445d-8de2-0689b91924b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:41.844025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de315ee7-e4ac-4954-b2ea-b6c437162b3d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:41.869846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:41.890855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:41.891967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:41.892598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:41.893155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:41.901657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:41.903720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:41.919078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:41.919988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:41.920861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:41.920861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:41.920861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:41.920861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:41.920861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:41.920861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:41.920861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:41.920861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:41.928481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 264/531] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:41.929860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:41.959906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64669a-ea95-445d-8de2-0689b91924b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:41.963135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64669a-ea95-445d-8de2-0689b91924b3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:42.053193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b2cf5-0857-4eec-8625-297efd0b56e0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:42.093950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:42.094997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:42.095487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:42.096568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:42.097514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:42.098227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:42.100050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:42.114296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 268/532] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:42.115448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64669a-ea95-445d-8de2-0689b91924b3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:42.117162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 265/533] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:42.118469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:42.125302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:42.175808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:42.233880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64669a-ea95-445d-8de2-0689b91924b3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:42.264745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39a832-01da-4a8d-bfe6-ae66f9a28d2d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:42.273730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:42.274527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:42.274847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:42.275218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:42.276031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:42.276455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:42.278008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:42.303902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 269/534] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:42.303902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:42.331403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:42.332274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:42.332616 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:42.332897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:42.347020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:42.347832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:42.366400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:42.368155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:42.370510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:42.370510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:42.370510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:42.372286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 266/535] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:42.375630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:42.418204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:42.457657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66831c74-e3cc-4556-b51a-eb8011eb3881 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:42.503155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:42.503989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:42.504296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:42.504569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:42.505309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=2550e85fa53d416887bff88fbff937d5) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:42.505615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:42.508420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:42.516530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 270/536] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:42 2022] GET /identity/v3/role_assignments?user.id=2550e85fa53d416887bff88fbff937d5&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:42.518379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:42.555635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:42.573294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ab1f2-8bec-4390-a956-2145232d244f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:42.630635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085ab7c-9732-4ef1-9602-ee213c3c1974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:42.631282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085ab7c-9732-4ef1-9602-ee213c3c1974 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:42.631637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085ab7c-9732-4ef1-9602-ee213c3c1974 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:42.631988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085ab7c-9732-4ef1-9602-ee213c3c1974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:42.632910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085ab7c-9732-4ef1-9602-ee213c3c1974 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=d5b5c6fcfde34c05a2fec0659dfd54a9) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:42.633353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085ab7c-9732-4ef1-9602-ee213c3c1974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:42.634936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085ab7c-9732-4ef1-9602-ee213c3c1974 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:42.642286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 267/537] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:42 2022] GET /identity/v3/role_assignments?user.id=d5b5c6fcfde34c05a2fec0659dfd54a9&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:42.644258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aac31f-f798-4d07-b537-24a8f50b0bfe None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:42.644642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aac31f-f798-4d07-b537-24a8f50b0bfe None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:42.645001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aac31f-f798-4d07-b537-24a8f50b0bfe None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:42.651274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 268/538] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:42.652841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3722af-557a-47e9-9e3d-11694c70990d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:42.653206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3722af-557a-47e9-9e3d-11694c70990d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:42.653583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3722af-557a-47e9-9e3d-11694c70990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:42.694685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cb3722af-557a-47e9-9e3d-11694c70990d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:42.696068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cb3722af-557a-47e9-9e3d-11694c70990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:42.756551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35090b-9fe5-4665-adb7-befef2d0d6cf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:42.840697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 269/539] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:42.843301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:42.854933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:42.856091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:42.856603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:42.857081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:42.860854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:42.860854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:42.861862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:42.888095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 271/540] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:42.897281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3722af-557a-47e9-9e3d-11694c70990d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:43.099229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:43.121593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3722af-557a-47e9-9e3d-11694c70990d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:43.198429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:43.199340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:43.199910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:43.200314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:43.218576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:43.222000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:43.252992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:43.254842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:43.256489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:43.256489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:43.256489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:43.256489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:43.256489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:43.256489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:43.256489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:43.256489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:43.264251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 270/541] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:43.267069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:43.319304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:43.437383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5467dc-243f-42f7-9a68-21af7bf5cdb7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:43.521750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:43.522769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:43.523250 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:43.523786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:43.525128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:43.525655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:43.528418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:43.570581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 272/542] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:43.578147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:43.604615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2508d-1cb9-48c4-86d1-6b2b8f4a6ad0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:43.675077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:43.694624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdb9ff-7d8b-4830-b9ec-2d5307bb75cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:43.695320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb9ff-7d8b-4830-b9ec-2d5307bb75cf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:43.695605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb9ff-7d8b-4830-b9ec-2d5307bb75cf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:43.695879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb9ff-7d8b-4830-b9ec-2d5307bb75cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:43.733761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdb9ff-7d8b-4830-b9ec-2d5307bb75cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:43.750854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7633f39-6d56-47df-9a47-74fedba0e286 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-97cdb9ff-7d8b-4830-b9ec-2d5307bb75cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:43.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:43.764761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:43.767214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:43.772146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 271/543] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:43.782102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdb9ff-7d8b-4830-b9ec-2d5307bb75cf None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:43.791111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:43.791940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:43.792380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:43.792804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:43.793751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=18ca375f53414475b75dae308cd19fef) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:43.800235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:43.800235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:43.803438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:43.807055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 273/544] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:43 2022] GET /identity/v3/role_assignments?user.id=18ca375f53414475b75dae308cd19fef&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:44.014890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdb9ff-7d8b-4830-b9ec-2d5307bb75cf None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:44.066897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:44.067747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:44.068213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:44.068648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:44.069466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:44.069909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:44.077670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:44.088608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 272/545] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:44.095107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:44.275767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636fa0d-ca62-48cb-972f-d1b2c571fd97 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:44.294121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:44.326317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:44.326880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:44.327169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:44.327447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:44.347867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:44.349393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:44.367537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:44.368632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:44.369564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:44.369564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:44.369564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:44.369564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 274/546] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:44.376222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:44.482225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:44.530364 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d4a26-5fc5-40d6-915c-303b8c6c8d14 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:44.586274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd55b04-b751-47bf-92a0-44c4a1e7326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:44.587103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd55b04-b751-47bf-92a0-44c4a1e7326f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:44.587389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd55b04-b751-47bf-92a0-44c4a1e7326f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:44.587664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd55b04-b751-47bf-92a0-44c4a1e7326f None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:44.588450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd55b04-b751-47bf-92a0-44c4a1e7326f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:44.588683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd55b04-b751-47bf-92a0-44c4a1e7326f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:44.590061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd55b04-b751-47bf-92a0-44c4a1e7326f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:44.609543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 273/547] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:44.615815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:44.679197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e1840-7c9d-4d4e-8ec7-802d4fe6bdad None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:44.679622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e1840-7c9d-4d4e-8ec7-802d4fe6bdad None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:44.679776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e1840-7c9d-4d4e-8ec7-802d4fe6bdad None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:44.681449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 274/548] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:18:44.687066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a6c2-2724-4b5d-8c5e-15bb04d29e96 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:44.687329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a6c2-2724-4b5d-8c5e-15bb04d29e96 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:44.687882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a6c2-2724-4b5d-8c5e-15bb04d29e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:44.707040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9942a6c2-2724-4b5d-8c5e-15bb04d29e96 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:44.708355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9942a6c2-2724-4b5d-8c5e-15bb04d29e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:44.893240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c9829-a8a4-4f17-9208-3afd53fab990 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:44.900322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 275/549] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:44.965963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87634191-dd15-49b5-9427-c993779218b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:44.967260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634191-dd15-49b5-9427-c993779218b7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:44.967875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634191-dd15-49b5-9427-c993779218b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:44.968546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87634191-dd15-49b5-9427-c993779218b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:44.982797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87634191-dd15-49b5-9427-c993779218b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:44.985602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87634191-dd15-49b5-9427-c993779218b7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-87634191-dd15-49b5-9427-c993779218b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:45.010559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:45.012659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:18:45.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:45.018382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 275/550] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:18:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:45.028306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a6c2-2724-4b5d-8c5e-15bb04d29e96 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:45.141708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188612-7e92-4e3b-8a6e-653751508db0 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:45.144459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188612-7e92-4e3b-8a6e-653751508db0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:45.145618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188612-7e92-4e3b-8a6e-653751508db0 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:45.147718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:45.150249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 276/551] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:45.160165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf10b37-7258-4b05-8192-05ea8f38b20c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:45.168469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf10b37-7258-4b05-8192-05ea8f38b20c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:45.168469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf10b37-7258-4b05-8192-05ea8f38b20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:45.206860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-edf10b37-7258-4b05-8192-05ea8f38b20c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:45.208081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-edf10b37-7258-4b05-8192-05ea8f38b20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:45.245876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a6c2-2724-4b5d-8c5e-15bb04d29e96 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:45.261309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:45.293978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:45.294599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:45.294883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:45.295160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:45.295899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:45.296222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:45.298382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:45.343953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 276/552] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 13:18:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:45.352049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:45.383764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 277/553] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:45.476448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b0db5-b51b-4076-bfc2-39ad1ebee49e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:45.491799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:45.512447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f09c71-f03f-4e37-a756-b9e27df4b52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:45.513068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f09c71-f03f-4e37-a756-b9e27df4b52c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:45.513352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f09c71-f03f-4e37-a756-b9e27df4b52c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:45.513628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f09c71-f03f-4e37-a756-b9e27df4b52c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:45.540687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf10b37-7258-4b05-8192-05ea8f38b20c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:45.597201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f09c71-f03f-4e37-a756-b9e27df4b52c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:45.598865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f09c71-f03f-4e37-a756-b9e27df4b52c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:45.720573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 277/554] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:18:45.728972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f09c71-f03f-4e37-a756-b9e27df4b52c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:45.747764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf10b37-7258-4b05-8192-05ea8f38b20c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:45.768898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:45.875572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55991142-0ca2-426c-8bb5-c02231e51bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:45.876237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55991142-0ca2-426c-8bb5-c02231e51bbb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:45.876527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55991142-0ca2-426c-8bb5-c02231e51bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:45.876806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55991142-0ca2-426c-8bb5-c02231e51bbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:45.880951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f09c71-f03f-4e37-a756-b9e27df4b52c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:45.903708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55991142-0ca2-426c-8bb5-c02231e51bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-55991142-0ca2-426c-8bb5-c02231e51bbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:45.909520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:45.910511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:45.911317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 278/555] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:45.926030 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55991142-0ca2-426c-8bb5-c02231e51bbb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:45.944353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:45.945126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:45.945415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:45.945729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:45.959000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:45.960528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:45.978672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:45.979721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:45.980705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:45.980705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:45.980705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:45.980705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 278/556] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:45.999144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:46.001032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.120570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:46.123210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55991142-0ca2-426c-8bb5-c02231e51bbb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.152258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeff4c-c8e3-4cb5-ba05-bd194ab26ab1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.161655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.162324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.162602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.162870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.163549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:46.163844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:46.165368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:46.179701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 279/557] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:46.180727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.238477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.239432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.239938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.240480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.241699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:46.242245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:46.244929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:46.267043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 279/558] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:46.267261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.277613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:46.303604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347a17a-5f43-4f1a-85fc-f05b204670a3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.344375 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df46eb8-d90b-4d21-af66-61a3360dd2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.345200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46eb8-d90b-4d21-af66-61a3360dd2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.345583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46eb8-d90b-4d21-af66-61a3360dd2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.345971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df46eb8-d90b-4d21-af66-61a3360dd2b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.357557 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df46eb8-d90b-4d21-af66-61a3360dd2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4df46eb8-d90b-4d21-af66-61a3360dd2b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:46.362862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:46.369395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 280/559] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:46.370361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2d0e-2d42-41eb-9fea-1434f0d29977 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.372682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2d0e-2d42-41eb-9fea-1434f0d29977 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.372682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e2d0e-2d42-41eb-9fea-1434f0d29977 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.372682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 281/560] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:46.372682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e2d0e-2d42-41eb-9fea-1434f0d29977 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.404811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf6a24-fee5-486c-979d-a2a4bcb9eec9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.435018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:46.445709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6792fa-b403-4869-9ba6-a417eba99649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.446341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6792fa-b403-4869-9ba6-a417eba99649 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.446770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6792fa-b403-4869-9ba6-a417eba99649 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.447046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6792fa-b403-4869-9ba6-a417eba99649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.461094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6792fa-b403-4869-9ba6-a417eba99649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:46.462918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6792fa-b403-4869-9ba6-a417eba99649 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1f6792fa-b403-4869-9ba6-a417eba99649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:46.482851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:46.484165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:46.485248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:46.485248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:46.485248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:46.489026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 280/561] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:46.489539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e2d0e-2d42-41eb-9fea-1434f0d29977 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.490304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3e116-bc8c-4b1c-adb1-7a141fd277b4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.490604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3e116-bc8c-4b1c-adb1-7a141fd277b4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.490879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3e116-bc8c-4b1c-adb1-7a141fd277b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.513925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f9f3e116-bc8c-4b1c-adb1-7a141fd277b4 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:46.515563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f9f3e116-bc8c-4b1c-adb1-7a141fd277b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:46.530425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.531049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.531424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.531601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.539084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:46.540567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:46.542473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:46.553690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:46.554643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 282/562] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:46.555584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.602376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 281/563] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:46.603767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3e116-bc8c-4b1c-adb1-7a141fd277b4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.705247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf012e-865b-4c07-9cc7-00f695fc2212 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.728564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3e116-bc8c-4b1c-adb1-7a141fd277b4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.741475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:46.747576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.748218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.748793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.748963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.749929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:46.750236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:46.751673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:46.765535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 283/564] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:46.765535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.772322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34873e1f-693d-4a3f-83c9-654ab3fc0de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.772938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34873e1f-693d-4a3f-83c9-654ab3fc0de4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.773218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34873e1f-693d-4a3f-83c9-654ab3fc0de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.773493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34873e1f-693d-4a3f-83c9-654ab3fc0de4 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.774195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34873e1f-693d-4a3f-83c9-654ab3fc0de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:46.774833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34873e1f-693d-4a3f-83c9-654ab3fc0de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:46.776821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34873e1f-693d-4a3f-83c9-654ab3fc0de4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:46.793573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 282/565] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:46.793573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ac6b6-b5f6-4d13-975a-8a329c2951da None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.793573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ac6b6-b5f6-4d13-975a-8a329c2951da None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.793573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ac6b6-b5f6-4d13-975a-8a329c2951da None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.793573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 283/566] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:46.793573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ac6b6-b5f6-4d13-975a-8a329c2951da None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.841084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:46.889162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294298e-b457-4e29-a064-3e0d53b57f9c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.913530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ac6b6-b5f6-4d13-975a-8a329c2951da None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:46.927788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.928431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.928719 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.928989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.929774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:46.930032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:46.931480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:46.944025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 284/567] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:46.944889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:46.959462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1523941-e557-4796-85ae-d1da77bd40c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:46.960112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1523941-e557-4796-85ae-d1da77bd40c1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.960425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1523941-e557-4796-85ae-d1da77bd40c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.960702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1523941-e557-4796-85ae-d1da77bd40c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:46.973575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1523941-e557-4796-85ae-d1da77bd40c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:46.974981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1523941-e557-4796-85ae-d1da77bd40c1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d1523941-e557-4796-85ae-d1da77bd40c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:46.989827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:46.990917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:46.991970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:46.991970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:46.991970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:46.991970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 284/568] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:46.992248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147687d-8b8a-4433-9540-7dd1ba5d34a0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:46.992734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147687d-8b8a-4433-9540-7dd1ba5d34a0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:46.992734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147687d-8b8a-4433-9540-7dd1ba5d34a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.011450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a147687d-8b8a-4433-9540-7dd1ba5d34a0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:47.012739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a147687d-8b8a-4433-9540-7dd1ba5d34a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:47.029603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:47.055267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f306f0-a031-4b05-8c84-8ed660dd7eca None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.076628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:47.091859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.098335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 285/569] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:47.098547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.098547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.098547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.098787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147687d-8b8a-4433-9540-7dd1ba5d34a0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.105142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:47.106736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:47.120149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:47.121863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:47.123129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 285/570] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:18:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:47.126574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.216036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147687d-8b8a-4433-9540-7dd1ba5d34a0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.246036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d31642-4f8e-4f32-8c04-311d0baa5ece None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.255769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4939adc-a265-4c3f-a473-ee6d06579499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.256478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4939adc-a265-4c3f-a473-ee6d06579499 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.256765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4939adc-a265-4c3f-a473-ee6d06579499 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.257037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4939adc-a265-4c3f-a473-ee6d06579499 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.270513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4939adc-a265-4c3f-a473-ee6d06579499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b4939adc-a265-4c3f-a473-ee6d06579499 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:47.275527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:47.276558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:47.277386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 286/571] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:47.278076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4939adc-a265-4c3f-a473-ee6d06579499 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.295803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:47.297363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.298254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.298530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.298841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.299818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=bd886328304543828dca35df3a0b7411) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:47.300588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:47.302075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:47.308204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 286/572] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/role_assignments?user.id=bd886328304543828dca35df3a0b7411&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:47.309084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.378955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:47.425178 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4939adc-a265-4c3f-a473-ee6d06579499 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.451797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e89923-13ff-4338-a543-48f7fd275c90 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.467791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.468521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.468814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.469092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.469758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:47.470063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:47.471720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:47.488769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 287/573] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:47.489628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.496408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.496953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.497278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.497495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.498227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:47.498527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:47.499994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:47.521476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 287/574] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:47.522078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.584219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:47.622021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2682b-fe7b-4b76-bd5c-7705c7b1d93b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.655968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338805d9-12e3-4c52-8f52-a0986606c670 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.657912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:47.661104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.661805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.662091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.662367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.663059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:47.663364 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:47.664822 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:47.673810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 288/575] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:47.674879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.701350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb58679-4c30-4aad-99fb-4cf4a5a27f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.701986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb58679-4c30-4aad-99fb-4cf4a5a27f95 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.702265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb58679-4c30-4aad-99fb-4cf4a5a27f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.702537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb58679-4c30-4aad-99fb-4cf4a5a27f95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.713551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb58679-4c30-4aad-99fb-4cf4a5a27f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-edb58679-4c30-4aad-99fb-4cf4a5a27f95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:47.718258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:47.719385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:47.720245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 288/576] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:47.721671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb58679-4c30-4aad-99fb-4cf4a5a27f95 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.805979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9ad18-f07e-42a9-9c32-ebbb71725a6f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.841496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:47.847232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.847867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.848156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.848300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb58679-4c30-4aad-99fb-4cf4a5a27f95 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:47.848761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.857060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:47.858887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:47.871132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:47.872143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 289/577] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:47.873132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.890960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055388b0-97e8-4532-93c6-688208240337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:47.891596 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055388b0-97e8-4532-93c6-688208240337 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:47.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055388b0-97e8-4532-93c6-688208240337 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:47.892157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055388b0-97e8-4532-93c6-688208240337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:47.906574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055388b0-97e8-4532-93c6-688208240337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:47.908732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055388b0-97e8-4532-93c6-688208240337 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-055388b0-97e8-4532-93c6-688208240337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:47.932794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:47.933791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:47.934718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:47.934718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:47.934718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:47.934718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 289/578] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:47.934718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055388b0-97e8-4532-93c6-688208240337 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:47.956130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:48.012053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb19ce3-c00f-4f78-a2a0-8be2532dc0bc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.055504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.056136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.056446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.056779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.062625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055388b0-97e8-4532-93c6-688208240337 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.068067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.069587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:48.084566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:48.090245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:48.091394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:48.091394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:48.091394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:48.091394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 290/579] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:48.091394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.115032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.115705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.115922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.116217 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.116896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:48.117201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.118579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.131534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 290/580] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:48.131534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.146865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:48.213621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78c3ae-4e3a-482f-b75b-cde7e3acacc7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.223438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:48.249559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51924fe3-ef89-4fe2-92ae-ae36d1755c35 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.254707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.255340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.255628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.256090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.256884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:48.257192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.259322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.284521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 291/581] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:48.285311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.289345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.289969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.290263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.290543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.291203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:48.291503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.292937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.311348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 291/582] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:48.312226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.412893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439afa0-81d1-44f7-83f2-4d4a0d0a238e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.415001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:48.433943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a04743-b078-4cbe-8044-0121ce1b00db None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.465108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.465721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.465997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.466266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.466927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:48.467286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.468971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.477713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.478375 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.478659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.478935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.482526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 292/583] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:48.483185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.484859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:48.496408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.498279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:48.520373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:48.521653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:48.522580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:48.522580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:48.522580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:48.529663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 292/584] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:48.529663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.597948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd97c4-83a3-496c-a3a3-3aeeff016b97 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.635411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.636038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.636347 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.636618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.646835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03825e-4d12-4081-9b3a-cca4ae18b690 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.650606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.653743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:48.669346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:48.670472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:48.671968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:48.671968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:48.671968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:48.671968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 293/585] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:48.671968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.688720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.689936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.690081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:48.690229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.690485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.697638 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.699103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:48.711415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:48.712447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:48.713348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:48.713348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:48.713348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:48.713348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:48.713348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:48.713348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:48.713348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:48.713348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:48.717357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 293/586] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:48.717970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.774080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:48.800541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722928e0-fd53-4dfc-9d77-949b72f19ec1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.837668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b95d4e-3c00-43e0-89a9-6d380a22d230 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:48.841043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.841702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.841984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.842255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.854532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.856114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:48.871772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:48.872995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:48.874117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 294/587] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:48.874817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.886454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:48.887130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:48.887345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:48.887671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:48.888357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:48.888660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:48.890067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:48.903722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 294/588] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:48.904627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:48.981095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:49.021196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da4921-9fd5-450e-a318-79163dffe1e5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.041901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ecbd4a-f3bc-49dc-865b-e2397606d636 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.055637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:49.063869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.065321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.065741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.066086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.066874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:49.067391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.068953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.082740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 295/589] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:49.085478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.086155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.086432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.086736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.087525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:49.087835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.088529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:49.089427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.102624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 295/590] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:49.103696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:49.213590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62a61a-12e2-47b3-8dd6-95ffb4061bf4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.232996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf06e9-817b-47db-9d51-f92a127608d6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.263038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.266371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.266371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.266371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.266371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:49.266371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.267034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.269296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:49.278499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.279376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.279811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.280268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.290096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.300990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 296/591] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 13:18:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:49.301186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.301287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:49.323397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:49.331111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 296/592] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:49.332026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:49.362974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:49.415720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e9f83-2592-4938-8cd1-036d50cf9d34 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.443100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83e296-9b84-48a1-861b-84b6e88841d2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.455103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.455913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.456103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.456419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.463475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.465144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:49.477544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:49.478654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 297/593] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:49.479705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:49.482149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b055b-e297-477d-9e14-182e084faa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.482897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b055b-e297-477d-9e14-182e084faa6d None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.483328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b055b-e297-477d-9e14-182e084faa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.483669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b055b-e297-477d-9e14-182e084faa6d None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/40561a306123461d92bc669be3962db3/roles/8760c26505544fdda1848788d4df9b5a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.500640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b055b-e297-477d-9e14-182e084faa6d None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=40561a306123461d92bc669be3962db3, role_id=8760c26505544fdda1848788d4df9b5a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.502396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b055b-e297-477d-9e14-182e084faa6d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.536875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:49.588607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a09a48-affd-4a83-b077-cda75aa6157b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.615070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 297/594] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:18:49 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/40561a306123461d92bc669be3962db3/roles/8760c26505544fdda1848788d4df9b5a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:49.617347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6990633-87f4-4036-9533-79f8b29b6de0 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.617838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6990633-87f4-4036-9533-79f8b29b6de0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.618305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6990633-87f4-4036-9533-79f8b29b6de0 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.625167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 298/595] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:49.625167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6990633-87f4-4036-9533-79f8b29b6de0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:49.639575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.640261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.640596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.640881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.655174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.657177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:49.681363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:49.682432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:49.683329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:49.683329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:49.683329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:49.688654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 298/596] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:49.691109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:49.747087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:49.776598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6990633-87f4-4036-9533-79f8b29b6de0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.822288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56ab39-49aa-4834-9067-003ecf31c8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.822541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f833fc-6cbd-4dac-9009-da057c11c94d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:49.823345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56ab39-49aa-4834-9067-003ecf31c8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.823631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56ab39-49aa-4834-9067-003ecf31c8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.823909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56ab39-49aa-4834-9067-003ecf31c8e2 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.824689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56ab39-49aa-4834-9067-003ecf31c8e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:49.825001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56ab39-49aa-4834-9067-003ecf31c8e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.826465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56ab39-49aa-4834-9067-003ecf31c8e2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.827034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:49.852262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 299/597] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 13:18:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:49.853819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bcd6b-3733-4e23-af8b-17c0bb713df2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.854112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bcd6b-3733-4e23-af8b-17c0bb713df2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.854391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bcd6b-3733-4e23-af8b-17c0bb713df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.865703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:49.866966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:49.867353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:49.867811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:49.868599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:49.868918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:49.870366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:49.874637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3a0bcd6b-3733-4e23-af8b-17c0bb713df2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:49.875947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3a0bcd6b-3733-4e23-af8b-17c0bb713df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:49.889345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 299/598] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:49.889345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:49.957957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 300/599] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:49.959064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bcd6b-3733-4e23-af8b-17c0bb713df2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.007426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74cb44-2a2e-4cd4-b4ce-2e361ef0c9fe None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.046044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:50.056060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.062255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.062255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.062255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.062255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:50.062255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:50.062255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:50.084348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 300/600] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:50.086141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.109237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bcd6b-3733-4e23-af8b-17c0bb713df2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.121788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:50.164325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.165541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.166200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.166580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.181302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:50.183517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:50.205677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:50.206793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:50.207766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:50.207766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:50.207766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:50.210939 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37846b-2174-43dc-b1d3-835a6b7d3f2c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.214456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 301/601] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:50.216677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.253182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25437275-8d7b-4c74-8976-f7b903f786c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.253979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25437275-8d7b-4c74-8976-f7b903f786c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.254260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25437275-8d7b-4c74-8976-f7b903f786c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.254532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25437275-8d7b-4c74-8976-f7b903f786c1 None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/2550e85fa53d416887bff88fbff937d5/roles/8760c26505544fdda1848788d4df9b5a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.276390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25437275-8d7b-4c74-8976-f7b903f786c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=2550e85fa53d416887bff88fbff937d5, role_id=8760c26505544fdda1848788d4df9b5a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:50.279078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25437275-8d7b-4c74-8976-f7b903f786c1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:50.335500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad37677-97a2-4356-98db-34a2ed593e8b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.353606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:50.371430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.372056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.372372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.372654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.379998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:50.381927 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:50.392590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 301/602] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:18:50 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/2550e85fa53d416887bff88fbff937d5/roles/8760c26505544fdda1848788d4df9b5a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:50.392590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25437275-8d7b-4c74-8976-f7b903f786c1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:50.418415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:50.419421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 302/603] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:50.420493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.495432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:50.526528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25437275-8d7b-4c74-8976-f7b903f786c1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.541228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38645-434c-4ef6-b199-907c37de0233 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.584698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.585699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.586607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.587047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.587970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:50.588591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:50.590188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:50.598430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a431fa-88e1-460c-a801-bf64d08dfa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.599189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a431fa-88e1-460c-a801-bf64d08dfa7c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.599552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a431fa-88e1-460c-a801-bf64d08dfa7c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.599907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a431fa-88e1-460c-a801-bf64d08dfa7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.607550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 302/604] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:50.608780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.618267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a431fa-88e1-460c-a801-bf64d08dfa7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-80a431fa-88e1-460c-a801-bf64d08dfa7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:50.628033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:50.629365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:50.630472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:50.631705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 303/605] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:50.635410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a431fa-88e1-460c-a801-bf64d08dfa7c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.689988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:50.734313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a128-38a1-4c29-94b6-28ec97711099 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.775667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.776316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.776659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.776878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.777612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:50.777995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:50.779493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:50.794070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 303/606] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:50.794958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.798225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:50.812997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a431fa-88e1-460c-a801-bf64d08dfa7c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.858938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.859831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.860350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.861118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.869509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:50.871267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:50.885936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:50.887081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:50.888284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:50.888284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:50.888284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:50.888284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:50.888284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:50.888284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:50.888284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:50.888284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:50.890621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 304/607] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:50.892380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:50.932478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6fbb1-b6bc-4352-8b15-a82a76e00141 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:50.988339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:50.988963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:50.989245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:50.989519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:50.990202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:50.990504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:50.998612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:51.017547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:51.022541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614f339-05ec-49a6-b472-4979a9936029 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:51.024037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 304/608] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:51.025421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:51.072072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c10194-c6bd-413c-a9d0-41c86cd426cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:51.073138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10194-c6bd-413c-a9d0-41c86cd426cc None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:51.073831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10194-c6bd-413c-a9d0-41c86cd426cc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:51.074321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c10194-c6bd-413c-a9d0-41c86cd426cc None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/d5b5c6fcfde34c05a2fec0659dfd54a9/roles/8760c26505544fdda1848788d4df9b5a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:51.099276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:51.109223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c10194-c6bd-413c-a9d0-41c86cd426cc None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=d5b5c6fcfde34c05a2fec0659dfd54a9, role_id=8760c26505544fdda1848788d4df9b5a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:51.113622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c10194-c6bd-413c-a9d0-41c86cd426cc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:51.176929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36d106-d5a3-48df-bfe6-cd7b885af46e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:51.215943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:51.217014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:51.217014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:51.217220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:51.218608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:51.218608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:51.219818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:51.231074 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 305/609] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:18:50 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/d5b5c6fcfde34c05a2fec0659dfd54a9/roles/8760c26505544fdda1848788d4df9b5a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:51.233086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c10194-c6bd-413c-a9d0-41c86cd426cc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:51.234582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 305/610] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:51.245603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:51.376735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b5cc2-fb78-42e2-892f-2320e51d93bd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:51.404043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:51.426968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c0497-beaa-453d-b6be-923a77be8b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:51.429082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0497-beaa-453d-b6be-923a77be8b14 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:51.429082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0497-beaa-453d-b6be-923a77be8b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:51.429082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0497-beaa-453d-b6be-923a77be8b14 None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/18ca375f53414475b75dae308cd19fef/roles/8760c26505544fdda1848788d4df9b5a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:51.449031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c0497-beaa-453d-b6be-923a77be8b14 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=18ca375f53414475b75dae308cd19fef, role_id=8760c26505544fdda1848788d4df9b5a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:51.450518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c0497-beaa-453d-b6be-923a77be8b14 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:51.455973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c10194-c6bd-413c-a9d0-41c86cd426cc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:51.499187 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:51.499901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:51.500208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:51.500495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:51.512823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:51.514275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:51.524503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:51.530483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 306/611] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:18:51 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/18ca375f53414475b75dae308cd19fef/roles/8760c26505544fdda1848788d4df9b5a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:51.534743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:51.544020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:51.545047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:51.545047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:51.545047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:51.545047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 306/612] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:51.547044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c0497-beaa-453d-b6be-923a77be8b14 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:51.766680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c0497-beaa-453d-b6be-923a77be8b14 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:51.805203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb90901-dafd-4bf4-8f8c-4f5d0e76e162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:51.806116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90901-dafd-4bf4-8f8c-4f5d0e76e162 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:51.806603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90901-dafd-4bf4-8f8c-4f5d0e76e162 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:51.807061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb90901-dafd-4bf4-8f8c-4f5d0e76e162 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:51.807888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb90901-dafd-4bf4-8f8c-4f5d0e76e162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:51.808384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb90901-dafd-4bf4-8f8c-4f5d0e76e162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:51.810004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb90901-dafd-4bf4-8f8c-4f5d0e76e162 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:51.823533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 307/613] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:51.828689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:51.957754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c12a3-25dc-4a13-ba8e-eb6ff29dbbf4 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:51.958305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c12a3-25dc-4a13-ba8e-eb6ff29dbbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:51.958305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c12a3-25dc-4a13-ba8e-eb6ff29dbbf4 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:51.959946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 308/614] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:51.960326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eff1ad-ef68-4b17-a979-4e9da323348d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:51.965409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee829329-d423-4da0-966e-ddd4c34a65e0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:51.965703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee829329-d423-4da0-966e-ddd4c34a65e0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:51.965998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee829329-d423-4da0-966e-ddd4c34a65e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:51.987738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ee829329-d423-4da0-966e-ddd4c34a65e0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:51.990013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ee829329-d423-4da0-966e-ddd4c34a65e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:52.001985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:52.003245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.003245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.003521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.012711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:52.014249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:52.021690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:52.028528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:52.029837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:52.031361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 307/615] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:18:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:52.037460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:52.084085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 309/616] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:18:52.130832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:52.174330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed395e4-a1f5-4661-862b-fa38d8edb158 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:52.225086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:52.226014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.226439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.226865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.227744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:52.228220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:52.229770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:52.256369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 308/617] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 13:18:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:52.258060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee829329-d423-4da0-966e-ddd4c34a65e0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:52.369938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee829329-d423-4da0-966e-ddd4c34a65e0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:52.409266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:52.410222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.410647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.411063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.417827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:52.425458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:52.427151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:52.447739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:52.448951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:52.450202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:52.450202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:52.450202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:52.451954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 310/618] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:52.457915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:52.476902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:52.603481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ccaf5-9422-419f-bbd5-56f90454baf9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:52.622471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:52.642097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68455108-910f-418f-b79b-215ee1ebdd9f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:52.681876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664396f5-faf0-4b27-9c21-53dbfaec55d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:52.682527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664396f5-faf0-4b27-9c21-53dbfaec55d9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.682809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664396f5-faf0-4b27-9c21-53dbfaec55d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.682977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:52.683085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664396f5-faf0-4b27-9c21-53dbfaec55d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.683698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.683986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.684305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.685062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:52.685371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:52.686861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:52.701320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 311/619] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:52.707221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:52.721692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664396f5-faf0-4b27-9c21-53dbfaec55d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-664396f5-faf0-4b27-9c21-53dbfaec55d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:52.726639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:52.727855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:52.728730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 309/620] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:18:52.737315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664396f5-faf0-4b27-9c21-53dbfaec55d9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:52.849254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:52.886585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749214ae-1a72-446b-bc8a-34646eb3f1ed None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:52.915908 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664396f5-faf0-4b27-9c21-53dbfaec55d9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:52.924762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf8079-caee-4493-8b1f-5d58e6fd71ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:52.925421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf8079-caee-4493-8b1f-5d58e6fd71ee None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.925702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf8079-caee-4493-8b1f-5d58e6fd71ee None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.925975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf8079-caee-4493-8b1f-5d58e6fd71ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.933662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf8079-caee-4493-8b1f-5d58e6fd71ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:52.935118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf8079-caee-4493-8b1f-5d58e6fd71ee None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-fadf8079-caee-4493-8b1f-5d58e6fd71ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:52.947794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:52.948804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:52.949687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 312/621] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:52.950628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57711-bc5c-473c-a871-c7b4ac7c8137 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.950795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57711-bc5c-473c-a871-c7b4ac7c8137 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.950948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da57711-bc5c-473c-a871-c7b4ac7c8137 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.957790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 313/622] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:52.959136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffbce6-9243-4d4c-baf8-f141e1359838 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.959428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffbce6-9243-4d4c-baf8-f141e1359838 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.959702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffbce6-9243-4d4c-baf8-f141e1359838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.973726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:52.974550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:52.974834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:52.975106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:52.975791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:52.976092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:52.977786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:52.978796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-69ffbce6-9243-4d4c-baf8-f141e1359838 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:52.980009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-69ffbce6-9243-4d4c-baf8-f141e1359838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:52.992353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 310/623] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:52.993282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:53.041499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:53.055580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 314/624] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:53.056990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffbce6-9243-4d4c-baf8-f141e1359838 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:53.116604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8b502-69b8-4ca1-b144-06c1e09b7e60 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:53.160273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:53.161202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:53.161644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:53.162074 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:53.162916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:53.163367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:53.165058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:53.179296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 311/625] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:53.185463 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:53.198861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:53.307702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:53.311184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefda151-24e8-486b-b2e6-f6110ae652ff None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:53.334203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffbce6-9243-4d4c-baf8-f141e1359838 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:53.359135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f742ae-1da7-4cf8-8a20-5b404e827ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:53.359814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f742ae-1da7-4cf8-8a20-5b404e827ab8 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:53.360093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f742ae-1da7-4cf8-8a20-5b404e827ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:53.360394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f742ae-1da7-4cf8-8a20-5b404e827ab8 None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/bd886328304543828dca35df3a0b7411/roles/8760c26505544fdda1848788d4df9b5a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:53.394922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f742ae-1da7-4cf8-8a20-5b404e827ab8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=bd886328304543828dca35df3a0b7411, role_id=8760c26505544fdda1848788d4df9b5a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:53.397051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f742ae-1da7-4cf8-8a20-5b404e827ab8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:53.401637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:53.402286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:53.402575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:53.402893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:53.420657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:53.420657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:53.433215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:53.434408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:53.435331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:53.435331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:53.435331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:53.435331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 315/626] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:53.435331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:53.479083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 312/627] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:18:53 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/bd886328304543828dca35df3a0b7411/roles/8760c26505544fdda1848788d4df9b5a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:18:53.483751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f742ae-1da7-4cf8-8a20-5b404e827ab8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:53.583598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf7695-ab5b-43e3-8570-c3775bc1b95e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:53.598444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:53.629735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fc12b-dc36-42af-aa09-0f06803e5509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:53.631226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fc12b-dc36-42af-aa09-0f06803e5509 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:53.632111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fc12b-dc36-42af-aa09-0f06803e5509 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:53.632608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fc12b-dc36-42af-aa09-0f06803e5509 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:53.648625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fc12b-dc36-42af-aa09-0f06803e5509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-fa5fc12b-dc36-42af-aa09-0f06803e5509 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:53.656091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:53.657327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:53.659130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:53.661761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 316/628] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:53.664586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6098634-fe5b-450d-9112-10ef7011bb0f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:53.664867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6098634-fe5b-450d-9112-10ef7011bb0f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:53.665144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6098634-fe5b-450d-9112-10ef7011bb0f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:53.666489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 317/629] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:53.667356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6098634-fe5b-450d-9112-10ef7011bb0f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:53.698784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f742ae-1da7-4cf8-8a20-5b404e827ab8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:53.747072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34cdc0-880b-4af0-bd7b-9de3262f04f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:53.747715 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cdc0-880b-4af0-bd7b-9de3262f04f3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:53.747991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cdc0-880b-4af0-bd7b-9de3262f04f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:53.748296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34cdc0-880b-4af0-bd7b-9de3262f04f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:53.748955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34cdc0-880b-4af0-bd7b-9de3262f04f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:53.749627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34cdc0-880b-4af0-bd7b-9de3262f04f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:53.750662 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34cdc0-880b-4af0-bd7b-9de3262f04f3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:53.763779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 313/630] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:53.765209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e126-9ab3-4823-b0dc-47d8e42e381e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:53.765557 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e126-9ab3-4823-b0dc-47d8e42e381e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:53.765904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e126-9ab3-4823-b0dc-47d8e42e381e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:53.789346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bac1e126-9ab3-4823-b0dc-47d8e42e381e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:53.790898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bac1e126-9ab3-4823-b0dc-47d8e42e381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:53.795348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:53.812827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6098634-fe5b-450d-9112-10ef7011bb0f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:53.860954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:53.861813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:53.862356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:53.862639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:53.863698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:53.864303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:53.865884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:53.875865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 314/631] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:53.877610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1e126-9ab3-4823-b0dc-47d8e42e381e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:53.878838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 318/632] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:53.885394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:53.930438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:54.015404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404f5db-5819-4120-88e8-ebae08161129 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.056124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.098389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.098389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.098389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.098389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:54.098389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:54.098389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:54.098389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:54.099086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:54.100134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:54.101412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 319/633] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:54.101412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:54.138392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1e126-9ab3-4823-b0dc-47d8e42e381e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.198838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:54.208654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.209460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.209742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.210015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.217714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018122c-e970-4d6f-9800-3ddcabcc9a70 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.219170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:54.222221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:54.225026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:54.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:54.242384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 315/634] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:18:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:54.243468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:54.264077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.265101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.265689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.266150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.267627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:54.268305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:54.269966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:54.284372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 320/635] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:18:54.289403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:54.393058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349c8f0-f092-4e9e-98a3-93023265810e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.423902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403836b0-79f0-42f0-9732-f300e7c23c6d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.440586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316b9f9-13ba-4cc7-9622-3f1bb57a7e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.441221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316b9f9-13ba-4cc7-9622-3f1bb57a7e4b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.441505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316b9f9-13ba-4cc7-9622-3f1bb57a7e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.441837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316b9f9-13ba-4cc7-9622-3f1bb57a7e4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.452599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316b9f9-13ba-4cc7-9622-3f1bb57a7e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-f316b9f9-13ba-4cc7-9622-3f1bb57a7e4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:54.459545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:54.460549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 316/636] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:54.462082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316b9f9-13ba-4cc7-9622-3f1bb57a7e4b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:54.465977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ea187-a44d-4de5-ac30-24d1061ee9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.466985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea187-a44d-4de5-ac30-24d1061ee9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.466985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea187-a44d-4de5-ac30-24d1061ee9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.467153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ea187-a44d-4de5-ac30-24d1061ee9e5 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.467846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea187-a44d-4de5-ac30-24d1061ee9e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:54.468151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea187-a44d-4de5-ac30-24d1061ee9e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:54.469612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ea187-a44d-4de5-ac30-24d1061ee9e5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:54.493828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 321/637] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:18:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:54.495355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fa4a4-530b-4aa6-8bd4-95d133c2cd5d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.495824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fa4a4-530b-4aa6-8bd4-95d133c2cd5d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.496280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fa4a4-530b-4aa6-8bd4-95d133c2cd5d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.497914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 322/638] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:54.498622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fa4a4-530b-4aa6-8bd4-95d133c2cd5d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:54.504285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:54.534234 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:54.578150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316b9f9-13ba-4cc7-9622-3f1bb57a7e4b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.616171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.616824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.617106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.617439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.618554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fa4a4-530b-4aa6-8bd4-95d133c2cd5d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.625445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:54.627540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:54.639539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:54.640912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:54.641874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 317/639] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:54.643037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:54.662457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748806b6-b902-4db2-a157-bcb7af3e9239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.663060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748806b6-b902-4db2-a157-bcb7af3e9239 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.663356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748806b6-b902-4db2-a157-bcb7af3e9239 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.663609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748806b6-b902-4db2-a157-bcb7af3e9239 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.664331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748806b6-b902-4db2-a157-bcb7af3e9239 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:54.664624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748806b6-b902-4db2-a157-bcb7af3e9239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:54.666005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748806b6-b902-4db2-a157-bcb7af3e9239 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:54.676111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 323/640] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:54.677361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca87d9-a288-4808-af00-58b89c53ee5b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.677646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca87d9-a288-4808-af00-58b89c53ee5b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.677926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca87d9-a288-4808-af00-58b89c53ee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.696627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-85ca87d9-a288-4808-af00-58b89c53ee5b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:54.697830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-85ca87d9-a288-4808-af00-58b89c53ee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:54.763268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:54.764999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdc07a-eb6a-4ca6-83c2-ee26ee944423 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.784210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 324/641] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:54.784210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca87d9-a288-4808-af00-58b89c53ee5b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:54.790379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:54.806714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5bd1a-47de-43d9-876f-8a840ac6b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.807338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5bd1a-47de-43d9-876f-8a840ac6b838 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.807685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5bd1a-47de-43d9-876f-8a840ac6b838 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.807970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5bd1a-47de-43d9-876f-8a840ac6b838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.903677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5bd1a-47de-43d9-876f-8a840ac6b838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:54.905737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5bd1a-47de-43d9-876f-8a840ac6b838 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:54.911283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca87d9-a288-4808-af00-58b89c53ee5b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:54.954053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:54.955424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:54.955808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:54.956219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:54.957590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:54.958003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:54.960013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:54.987335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 325/642] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 13:18:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:54.988566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.034163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 318/643] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:55.035231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5bd1a-47de-43d9-876f-8a840ac6b838 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.045781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:55.058949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:55.098457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f40988-3e3d-4d01-933c-8910364a6629 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.138549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.139353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.139695 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.139975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.144067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5bd1a-47de-43d9-876f-8a840ac6b838 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.154226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:55.156087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:55.173958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:55.180360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:55.181575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:55.181575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:55.181575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:55.181575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 326/644] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:55.181575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.192538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71ba48-76cd-42be-947f-f16cfb0de75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.193345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71ba48-76cd-42be-947f-f16cfb0de75e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.193631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71ba48-76cd-42be-947f-f16cfb0de75e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.193902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71ba48-76cd-42be-947f-f16cfb0de75e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.269075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71ba48-76cd-42be-947f-f16cfb0de75e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:55.270540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71ba48-76cd-42be-947f-f16cfb0de75e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:55.292512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd981b-017c-4433-a685-5bf0495edd61 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.294465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:55.297333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:55.331806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.332478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.332761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.333036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.343366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:55.344847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:55.360450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:55.362172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:55.363234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:55.363234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:55.363234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:55.363234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 327/645] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:55.363234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.388226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 319/646] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:55.388226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71ba48-76cd-42be-947f-f16cfb0de75e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.476979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43e2ff-efd2-4c14-bb18-874d7b32ef64 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.498665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71ba48-76cd-42be-947f-f16cfb0de75e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.514876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99f610-a989-4956-866a-990bf82d78c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.515667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f610-a989-4956-866a-990bf82d78c3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.515950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f610-a989-4956-866a-990bf82d78c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.516248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f610-a989-4956-866a-990bf82d78c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.521188 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:55.528238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99f610-a989-4956-866a-990bf82d78c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7c99f610-a989-4956-866a-990bf82d78c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:55.533519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:55.534738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 328/647] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:55.535578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99f610-a989-4956-866a-990bf82d78c3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.541356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.541976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.542268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.542529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.543182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:55.543485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:55.544914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:55.556651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:55.559104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 320/648] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:55.559931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.652655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99f610-a989-4956-866a-990bf82d78c3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.677055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ef649-6e7a-4345-be55-16413acee99e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.688653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.689290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.689566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.689838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.690478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:55.690777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:55.692141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:55.703822 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 329/649] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:55.705162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.712462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.713120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.713395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.713666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.725103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:55.726544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:55.740407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:55.741520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:55.742549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:55.742549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:55.742549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:55.742549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 321/650] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:55.742833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.777559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:55.792647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:55.820005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e627bef-fd18-47e8-8e78-2eaaa818405f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.849578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062c50c-eeb6-4cb1-b1f5-f61d22340177 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:55.857073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55bf70-ce9a-4b70-bb27-18899e939853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.857669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55bf70-ce9a-4b70-bb27-18899e939853 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.857947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55bf70-ce9a-4b70-bb27-18899e939853 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.858217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55bf70-ce9a-4b70-bb27-18899e939853 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.858871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55bf70-ce9a-4b70-bb27-18899e939853 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:55.859166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55bf70-ce9a-4b70-bb27-18899e939853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:55.860588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55bf70-ce9a-4b70-bb27-18899e939853 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:55.868990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 330/651] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:55.870155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6e9ff-e90f-4463-8cda-2e20bc3f4d44 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.870442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6e9ff-e90f-4463-8cda-2e20bc3f4d44 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.870711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6e9ff-e90f-4463-8cda-2e20bc3f4d44 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.875989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 331/652] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:55.877031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6e9ff-e90f-4463-8cda-2e20bc3f4d44 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.890971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:55.891607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:55.891885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:55.892846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:55.900935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:55.902387 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:55.914771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:55.915951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 322/653] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:55.917094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:55.984640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6e9ff-e90f-4463-8cda-2e20bc3f4d44 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.017816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.019375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.020007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.020320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.020596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.026617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86833f4a-1118-4ace-a18d-1162871d3f21 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.027445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.028921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.030657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:56.041177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:56.042326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 332/654] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:56.043463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.065561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92e831-b85d-4ef9-bbe9-d893e25f3e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.066171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e831-b85d-4ef9-bbe9-d893e25f3e1f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.066455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e831-b85d-4ef9-bbe9-d893e25f3e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.066729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92e831-b85d-4ef9-bbe9-d893e25f3e1f None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.067377 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92e831-b85d-4ef9-bbe9-d893e25f3e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:56.067677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92e831-b85d-4ef9-bbe9-d893e25f3e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.069097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92e831-b85d-4ef9-bbe9-d893e25f3e1f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.082154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 323/655] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:56.083366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413381d-68c2-44aa-a7e2-16c80e7d0e1c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.083657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413381d-68c2-44aa-a7e2-16c80e7d0e1c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.083933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413381d-68c2-44aa-a7e2-16c80e7d0e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.101935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7413381d-68c2-44aa-a7e2-16c80e7d0e1c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:56.103119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7413381d-68c2-44aa-a7e2-16c80e7d0e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:56.148006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7390ff-f4ce-4b66-9306-ec07f7bbc94c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.169803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 324/656] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:56.170758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413381d-68c2-44aa-a7e2-16c80e7d0e1c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.187982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.188725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.189719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.189972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.201556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.203026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:56.218182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:56.219179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:56.220093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:56.220093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:56.220093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:56.224925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 333/657] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:56.225790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.260966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.277936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.281192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413381d-68c2-44aa-a7e2-16c80e7d0e1c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.319262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.319999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.320367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.320703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.321496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:56.321866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.323373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.334064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafdd40-b69b-4a03-85fe-3a8b393bbb98 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.346643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 325/658] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:56.347562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.372024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.372664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.372932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.373204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.373899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:56.374203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.375632 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.388780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 334/659] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:56.389634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.467398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1563d15-f396-4c10-b927-776c52e79bf6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.504499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.505137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.505416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.505688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.508130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bada20-0030-47fa-a981-772f8de89245 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.513105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.514544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.518933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.519992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:56.527856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:56.528824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:56.529709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 326/660] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:56.530190 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.548913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.549298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.549612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.549882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.550519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:56.550817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.552169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.563489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 335/661] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:56.564582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.633218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d989b0-0220-49de-b68b-64ab3f096577 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.668418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614a80e-bcc8-48dd-b808-e7462626b2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.669022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614a80e-bcc8-48dd-b808-e7462626b2cf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.669307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614a80e-bcc8-48dd-b808-e7462626b2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.669581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614a80e-bcc8-48dd-b808-e7462626b2cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.676867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479971d3-579f-44b5-a02b-d80bec6bf023 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.712870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.713467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.713750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.714025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.714748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=40561a306123461d92bc669be3962db3) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:56.715058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.716525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.723075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 336/662] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/role_assignments?user.id=40561a306123461d92bc669be3962db3&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:56.723923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.739194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.742067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614a80e-bcc8-48dd-b808-e7462626b2cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:56.743693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614a80e-bcc8-48dd-b808-e7462626b2cf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:56.778090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.858035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7d520-6d12-429e-a884-ebfd62e637e1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:56.872231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 327/663] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:56.872893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614a80e-bcc8-48dd-b808-e7462626b2cf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.908439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a13de-54ee-4a1a-9961-6cef528175e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:56.909122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a13de-54ee-4a1a-9961-6cef528175e9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:56.909221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a13de-54ee-4a1a-9961-6cef528175e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:56.909475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a13de-54ee-4a1a-9961-6cef528175e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:56.920796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a13de-54ee-4a1a-9961-6cef528175e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e41a13de-54ee-4a1a-9961-6cef528175e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:56.925913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:56.926977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:56.927824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 337/664] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:56.929196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a13de-54ee-4a1a-9961-6cef528175e9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:56.996601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:56.999578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614a80e-bcc8-48dd-b808-e7462626b2cf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.036316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.037391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.037967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.038303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.039703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:57.040015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.041652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.041867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:57.050403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a13de-54ee-4a1a-9961-6cef528175e9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.060323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 328/665] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:57.060323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:57.089062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.089805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.090082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.090361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.098651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.100132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:57.114778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:57.117291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 338/666] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:18:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:57.118264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:57.182036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49e183-4b04-4697-8db4-c01f1ac432f6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.221900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.223361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.223922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.224373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.231546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c71590-397a-47d4-adcd-c97413cde63a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.239736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.241872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:57.257348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:57.258528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:57.259426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:57.259426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:57.259426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:57.259426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 329/667] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:57.259979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:57.275203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:57.276787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.277513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.277870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.278212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.278980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:57.279344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.280838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.296685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 339/668] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:57.297573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:57.321258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:57.382283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66121e0-b55a-41d7-a50e-33fbed77a710 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.429996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.431030 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.431492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.432316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.433480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=2550e85fa53d416887bff88fbff937d5) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:57.433954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.434929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eb776-1c05-4afb-b84c-1b5977f643e2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.435773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.445680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 330/669] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/role_assignments?user.id=2550e85fa53d416887bff88fbff937d5&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:57.447153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:57.479732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.480539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.480921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.481262 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.482085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:57.482546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.484276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.510303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 340/670] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:57.511108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:57.574954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:57.602818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff23d9a-704c-4718-81f4-b7ee0fc2ac23 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.624283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:57.649035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.650453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.650947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.651680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.652732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:57.653584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.655208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.656383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab580e-eb26-43d9-9668-40dbd49795a3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.669468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 331/671] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:57.670556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:57.698274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.698830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.699108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.699380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.720808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.723097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:57.745312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:57.746346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:57.747278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:57.747278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:57.747278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:57.750524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 341/672] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:57.752340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:57.792362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5b4a8-ca9a-4c75-bc82-30b3968abdba None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.831916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716938-83b0-4ed4-9487-8375285ee7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.834715 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716938-83b0-4ed4-9487-8375285ee7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.835057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716938-83b0-4ed4-9487-8375285ee7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.835336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716938-83b0-4ed4-9487-8375285ee7b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.840548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:57.888389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed53f-add7-4126-ad55-58bf5682918b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:57.925705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:57.931686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716938-83b0-4ed4-9487-8375285ee7b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.933860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716938-83b0-4ed4-9487-8375285ee7b8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.935007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:57.935643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:57.935927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:57.936224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:57.943623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:57.945235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:57.958071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:57.958990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:57.959880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:57.959880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:57.959880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:57.959880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:57.959880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:57.959880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:57.959880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:57.959880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:57.963994 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 342/673] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:57.964908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.071888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 332/674] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:58.072946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716938-83b0-4ed4-9487-8375285ee7b8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.083197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a08fb-a42e-4188-bb8a-08d34f8cf368 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:58.113724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:58.132747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:58.134125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.134878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.135367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.136409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:58.137164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:58.139119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:58.160773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 343/675] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:58.161832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.191395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716938-83b0-4ed4-9487-8375285ee7b8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:58.226309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:58.250858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:58.253162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.253591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.254298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.255149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:58.255782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:58.258228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:58.275353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 333/676] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:58.276506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.295651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2efb73-3eb9-4a70-8aff-d3b5de60f5a0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:58.344071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:58.345928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.346355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.347024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.359559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:58.362667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:58.379328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:58.380552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:58.381623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:58.381623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:58.381623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:58.382963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 344/677] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:58.383985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.396925 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:58.405598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386936e6-2852-44df-8b8e-759c42ff7f00 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:58.453651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:58.454853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.455310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.455720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.467210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:58.468864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:18:58.484525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:58.485787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:18:58.486782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:58.488921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 334/678] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:18:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:58.489862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.505038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:58.509239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690f8c0-5e73-4de4-b6b7-81e9623e6917 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:58.547413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:58.548487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.549465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.549982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.550699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=d5b5c6fcfde34c05a2fec0659dfd54a9) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:58.551088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:58.552651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:58.559005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 345/679] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:58 2022] GET /identity/v3/role_assignments?user.id=d5b5c6fcfde34c05a2fec0659dfd54a9&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:58.566062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.629136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c1af0-2b56-4e41-9965-9b8ffac4dc11 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:58.677086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:58.677976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.678423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.679005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.680269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:58.680909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:58.682788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:58.701082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 335/680] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:58.707937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.730347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:58.778569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6da5b-bc34-4972-b40f-127d6dae6faf None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:58.832997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867816f-2e02-41e2-bae1-85b144af2111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:58.832997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867816f-2e02-41e2-bae1-85b144af2111 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.833295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867816f-2e02-41e2-bae1-85b144af2111 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.833635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867816f-2e02-41e2-bae1-85b144af2111 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.834280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867816f-2e02-41e2-bae1-85b144af2111 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:58.837351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867816f-2e02-41e2-bae1-85b144af2111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:58.837351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867816f-2e02-41e2-bae1-85b144af2111 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:58.849534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01a93c-ffad-460f-a7c3-5ceb3cac4dd4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:58.859490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 346/681] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 13:18:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:58.861256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62feb91-59a3-439c-85ea-4ed05429858a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.862091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62feb91-59a3-439c-85ea-4ed05429858a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.863466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62feb91-59a3-439c-85ea-4ed05429858a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.864047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 347/682] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:58.869407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62feb91-59a3-439c-85ea-4ed05429858a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:58.895805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c5fd8-d786-403f-a4b1-bd9e74d66975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:58.896475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c5fd8-d786-403f-a4b1-bd9e74d66975 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.896744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c5fd8-d786-403f-a4b1-bd9e74d66975 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.897016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c5fd8-d786-403f-a4b1-bd9e74d66975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.904050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c5fd8-d786-403f-a4b1-bd9e74d66975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:58.905535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c5fd8-d786-403f-a4b1-bd9e74d66975 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:58.907836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-928c5fd8-d786-403f-a4b1-bd9e74d66975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:58.917658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:58.921356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 336/683] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84e83d-b5de-429e-926b-b2aa57955364 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:58.922240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84e83d-b5de-429e-926b-b2aa57955364 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:58.922846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84e83d-b5de-429e-926b-b2aa57955364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:58.942755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fc84e83d-b5de-429e-926b-b2aa57955364 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:58.944376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fc84e83d-b5de-429e-926b-b2aa57955364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:59.018571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62feb91-59a3-439c-85ea-4ed05429858a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:59.036277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 337/684] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:59.037243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84e83d-b5de-429e-926b-b2aa57955364 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:59.058619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:59.079432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ef81a-7ea3-49a6-8b04-76b4bd25a0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:59.080316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ef81a-7ea3-49a6-8b04-76b4bd25a0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.080668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ef81a-7ea3-49a6-8b04-76b4bd25a0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.080950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ef81a-7ea3-49a6-8b04-76b4bd25a0d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.189771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ef81a-7ea3-49a6-8b04-76b4bd25a0d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:59.191692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ef81a-7ea3-49a6-8b04-76b4bd25a0d0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:59.215569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:59.268081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84e83d-b5de-429e-926b-b2aa57955364 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:59.303108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 348/685] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:18:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:18:59.310292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ef81a-7ea3-49a6-8b04-76b4bd25a0d0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:59.350911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:59.351900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.354117 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.354605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.355521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:59.357401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:59.359073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:59.386826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 338/686] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:59.387793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:59.455768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ef81a-7ea3-49a6-8b04-76b4bd25a0d0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:59.460781 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:59.476880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:59.492192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:59.492867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.493103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.493379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.502345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3c38-0379-435c-be6e-f84bbf153af1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:59.505759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:59.506801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:18:59.521535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:59.522538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:18:59.523422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:18:59.523422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:18:59.523422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:18:59.523422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 349/687] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:59.524140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:59.545063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c879b2-0644-4a5f-b7f0-36a60a8e748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:59.546043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c879b2-0644-4a5f-b7f0-36a60a8e748a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.546530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c879b2-0644-4a5f-b7f0-36a60a8e748a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.547029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c879b2-0644-4a5f-b7f0-36a60a8e748a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.559622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c879b2-0644-4a5f-b7f0-36a60a8e748a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-85c879b2-0644-4a5f-b7f0-36a60a8e748a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:18:59.565673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:59.566639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:18:59.567467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:59.569668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 339/688] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:59.572228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c879b2-0644-4a5f-b7f0-36a60a8e748a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:59.637655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f47e1-7c4c-419b-87cf-d1e1a004a9c8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:59.675863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:59.676521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.676803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.677078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.677912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=18ca375f53414475b75dae308cd19fef) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:59.678225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:59.679754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:59.692220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 350/689] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/role_assignments?user.id=18ca375f53414475b75dae308cd19fef&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:59.692220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:59.701845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c879b2-0644-4a5f-b7f0-36a60a8e748a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:59.734191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:59.749218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:18:59.753932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee03a55-fa22-4e92-a495-396ba41a27f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:59.756552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee03a55-fa22-4e92-a495-396ba41a27f9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.756552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee03a55-fa22-4e92-a495-396ba41a27f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.756552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee03a55-fa22-4e92-a495-396ba41a27f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.756552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee03a55-fa22-4e92-a495-396ba41a27f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:59.756552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee03a55-fa22-4e92-a495-396ba41a27f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:59.758371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee03a55-fa22-4e92-a495-396ba41a27f9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:59.777999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 340/690] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:59.779830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b2e03-0988-4763-b0bc-dbd1621ad9e1 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.780384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b2e03-0988-4763-b0bc-dbd1621ad9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.780876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b2e03-0988-4763-b0bc-dbd1621ad9e1 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.783899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 341/691] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:18:59.785102 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b2e03-0988-4763-b0bc-dbd1621ad9e1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:59.812202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bd87b-de95-4ec3-973a-b76ce7d73bda None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:59.857230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8379b-5602-48fc-be5e-ef35f47bcf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:59.857866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8379b-5602-48fc-be5e-ef35f47bcf53 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.858149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8379b-5602-48fc-be5e-ef35f47bcf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.858426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8379b-5602-48fc-be5e-ef35f47bcf53 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.859478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8379b-5602-48fc-be5e-ef35f47bcf53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:18:59.859919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8379b-5602-48fc-be5e-ef35f47bcf53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:59.862124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8379b-5602-48fc-be5e-ef35f47bcf53 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:59.878383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 351/692] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:18:59.880694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589973f-dc27-4621-8cc7-45c8c704cbd0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.881801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589973f-dc27-4621-8cc7-45c8c704cbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.881801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589973f-dc27-4621-8cc7-45c8c704cbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.902043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3589973f-dc27-4621-8cc7-45c8c704cbd0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:18:59.903328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3589973f-dc27-4621-8cc7-45c8c704cbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:18:59.911161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b2e03-0988-4763-b0bc-dbd1621ad9e1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:18:59.957075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:18:59.957712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:18:59.957992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:18:59.958266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:18:59.965828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:18:59.967338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:18:59.980098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:18:59.981169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:18:59.982442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 342/693] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:18:59.982986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:18:59.997693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 352/694] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:18:59.998536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589973f-dc27-4621-8cc7-45c8c704cbd0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:00.057324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:00.081813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:00.165837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589973f-dc27-4621-8cc7-45c8c704cbd0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:00.181184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70741f9c-9b24-4aee-9ba0-00f49316cd99 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:00.208329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:00.209380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.209978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.210530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.211496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:00.212035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:00.213914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:00.231900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:00.231900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.231900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.231900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.232565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 353/695] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:00.234909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:00.250400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:00.253025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:00.278567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:00.279988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:00.280946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:00.280946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:00.280946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:00.284653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 343/696] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:00.291038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:00.378786 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:00.392782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3616d-2963-419a-93ad-631f8e7faad7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:00.419479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:00.425858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6166742-5400-4a62-9359-dc9f66bc5944 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:00.430860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:00.431602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.431952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.432398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.433205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=bd886328304543828dca35df3a0b7411) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:00.433583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:00.435255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:00.444291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 354/697] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:00 2022] GET /identity/v3/role_assignments?user.id=bd886328304543828dca35df3a0b7411&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:00.444291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:00.495385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:00.496021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.496357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.496635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.497296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:00.497601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:00.499015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:00.526507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 344/698] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:00.534565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:00.565608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1ffe8-ff5c-4a53-bbb0-fa90e0b8a253 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:00.635124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:00.635786 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.636082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.636381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.636983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:00.638607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:00.660209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:00.661314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:00.662157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:00.662910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 355/699] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 13:19:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:00.669933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e08476-59ea-4653-99fd-84f956d4a4a8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:00.670359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:00.701251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:00.711312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:00.722413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.722413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.722413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.722413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:00.724446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:00.738532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:00.739513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:00.740794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 345/700] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:19:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:00.745788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:00.769331 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:00.827590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c77036-ffa0-4b9e-bd5d-843ce5c075ff None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:00.867339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597c3d9-7e5b-4a73-a121-b598a4e79779 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:00.873686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4306ce-862e-4b0f-ab20-0a028ca4d93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:00.874301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4306ce-862e-4b0f-ab20-0a028ca4d93f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.875152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4306ce-862e-4b0f-ab20-0a028ca4d93f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.875152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4306ce-862e-4b0f-ab20-0a028ca4d93f None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.875482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4306ce-862e-4b0f-ab20-0a028ca4d93f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:00.875787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4306ce-862e-4b0f-ab20-0a028ca4d93f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:00.877289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4306ce-862e-4b0f-ab20-0a028ca4d93f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:00.886113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 356/701] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 13:19:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:00.888705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be68c59-5adf-4077-9781-d2a5b71d7669 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.889006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be68c59-5adf-4077-9781-d2a5b71d7669 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.889290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be68c59-5adf-4077-9781-d2a5b71d7669 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.890781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 357/702] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:00.898785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba4eba-3587-42e0-9d2c-52fefd0763fd None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.899265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba4eba-3587-42e0-9d2c-52fefd0763fd None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.899791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba4eba-3587-42e0-9d2c-52fefd0763fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.919666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:00.920354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:00.920646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:00.920922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:00.921288 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-55ba4eba-3587-42e0-9d2c-52fefd0763fd None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:00.921652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:00.921967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:00.922509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-55ba4eba-3587-42e0-9d2c-52fefd0763fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:00.924511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:00.945018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 346/703] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:19:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:00.946839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:01.016335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 358/704] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:19:01.018617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba4eba-3587-42e0-9d2c-52fefd0763fd None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:01.040747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:01.052558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:01.069459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32740fe9-1ac8-44b2-85ee-c160e4a11689 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:01.137461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:01.138453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:01.138931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:01.139422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:01.140374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:01.140874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:01.142651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:01.155704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 347/705] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 13:19:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:01.162286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba4eba-3587-42e0-9d2c-52fefd0763fd None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:01.206083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c04d-2a81-4304-9395-ac7a258459f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:01.206896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c04d-2a81-4304-9395-ac7a258459f8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:01.207399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c04d-2a81-4304-9395-ac7a258459f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:01.207715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292c04d-2a81-4304-9395-ac7a258459f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:01.297019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292c04d-2a81-4304-9395-ac7a258459f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:01.300057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292c04d-2a81-4304-9395-ac7a258459f8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:01.313004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:01.404495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:01.414557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 359/706] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 13:19:01.426996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c04d-2a81-4304-9395-ac7a258459f8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:01.550650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55499460-a64a-49ba-ab70-44bc10e3094f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:01.560247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292c04d-2a81-4304-9395-ac7a258459f8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:01.565649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:01.591705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:01.592376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:01.592660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:01.592947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:01.604708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:01.606196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:01.620907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:01.621904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:01.622394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:01.622642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:01.623532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:01.624435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:01.624435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:01.624435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:01.624435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 360/707] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:01.624996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:01.636130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:01.640427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:01.641868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:01.644031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:01.669265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:01.670269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:01.671792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:01.671792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:01.671792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:01.671792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 348/708] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:01.681245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:01.754918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92147d-6cb5-4418-b726-5aaf4ce4c08d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:01.806517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:01.807488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:01.807990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:01.808482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:01.809386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:01.809883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:01.811497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:01.825482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 361/709] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:01.834338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:01.883762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:01.897892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bd13d-d91c-43dd-932f-29b397789188 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:01.963118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:01.963658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:01.963931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:01.964223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:01.964874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:01.965171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:01.966553 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:01.993438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 349/710] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:01.997182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:02.027615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615118f9-b188-422f-b98d-495a634bee17 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:02.093075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:02.124915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c236b-4d9f-4633-b7b8-d719fdd9acad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:02.125564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c236b-4d9f-4633-b7b8-d719fdd9acad None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:02.125843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c236b-4d9f-4633-b7b8-d719fdd9acad None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:02.126145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c236b-4d9f-4633-b7b8-d719fdd9acad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:02.143166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c236b-4d9f-4633-b7b8-d719fdd9acad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:02.144660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c236b-4d9f-4633-b7b8-d719fdd9acad None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:02.147725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfd426-57ef-413f-8039-6a89142bdb6b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d53c236b-4d9f-4633-b7b8-d719fdd9acad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:02.157253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:02.159086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:02.159972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:02.159972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:02.159972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:02.159972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:02.159972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:02.159972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:02.159972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:02.159972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:02.164099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 362/711] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:02.165401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6614c-9696-4548-a9a6-944d6839b4f3 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:02.165681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6614c-9696-4548-a9a6-944d6839b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:02.165950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6614c-9696-4548-a9a6-944d6839b4f3 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:02.171651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 363/712] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:02.172418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6614c-9696-4548-a9a6-944d6839b4f3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:02.194718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e24cbf-9bd3-44f7-bea0-d70d94fe2f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:02.195754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e24cbf-9bd3-44f7-bea0-d70d94fe2f5e None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:02.196651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e24cbf-9bd3-44f7-bea0-d70d94fe2f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:02.197172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e24cbf-9bd3-44f7-bea0-d70d94fe2f5e None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:02.198271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e24cbf-9bd3-44f7-bea0-d70d94fe2f5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:02.200718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e24cbf-9bd3-44f7-bea0-d70d94fe2f5e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:02.258408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:02.282915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6614c-9696-4548-a9a6-944d6839b4f3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:02.299853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 350/713] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:19:01 2022] POST /identity/v3/users => generated 254 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:02.301422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae69e8-b961-445c-8f8b-c28ee04d8a11 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:02.301773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae69e8-b961-445c-8f8b-c28ee04d8a11 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:02.301999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae69e8-b961-445c-8f8b-c28ee04d8a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:02.341471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-62ae69e8-b961-445c-8f8b-c28ee04d8a11 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:02.342703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-62ae69e8-b961-445c-8f8b-c28ee04d8a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:02.360142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:02.360840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:02.361122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:02.361394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:02.362102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:02.362411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:02.363894 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:02.393960 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 364/714] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:02.402841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:02.505149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 351/715] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:19:02.563110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b750561-65ce-4df2-a6c7-67ee9b583d66 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:02.601183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e4aea-d3e6-40a5-8237-2b15ac46d41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:02.602268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e4aea-d3e6-40a5-8237-2b15ac46d41f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:02.602749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e4aea-d3e6-40a5-8237-2b15ac46d41f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:02.603273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e4aea-d3e6-40a5-8237-2b15ac46d41f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:02.604250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e4aea-d3e6-40a5-8237-2b15ac46d41f None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=40561a306123461d92bc669be3962db3) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:02.604783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e4aea-d3e6-40a5-8237-2b15ac46d41f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:02.606444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e4aea-d3e6-40a5-8237-2b15ac46d41f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:02.614355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 365/716] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:02 2022] GET /identity/v3/role_assignments?user.id=40561a306123461d92bc669be3962db3&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:02.813344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae69e8-b961-445c-8f8b-c28ee04d8a11 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:02.864871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:02.946268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae69e8-b961-445c-8f8b-c28ee04d8a11 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:02.957988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfa08f-52ab-4d92-945c-82a136301b6a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:02.958652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfa08f-52ab-4d92-945c-82a136301b6a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:02.959148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfa08f-52ab-4d92-945c-82a136301b6a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:02.960952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 366/717] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:02.966623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab64d7-e022-4f23-98b9-f71ebf40b636 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:02.967172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab64d7-e022-4f23-98b9-f71ebf40b636 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:02.967652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab64d7-e022-4f23-98b9-f71ebf40b636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:02.984344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:02.993116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-07ab64d7-e022-4f23-98b9-f71ebf40b636 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:02.994322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-07ab64d7-e022-4f23-98b9-f71ebf40b636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:03.002953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:03.003452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:03.003730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:03.004002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] PATH_INFO: `/v3/services/network` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:03.004629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:03.005992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:03.026460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:03.027601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:03.028730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 352/718] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:02 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:03.034562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:03.118726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 367/719] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:19:03.197873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3c4b5-d2b5-419c-89b1-a91567085fcc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:03.199577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:03.235643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:03.236583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:03.236971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:03.237469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:03.238105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] RBAC: Adding query filter params (name=network) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:03.238405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:03.239770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:03.244915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 353/720] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:03.249950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab64d7-e022-4f23-98b9-f71ebf40b636 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:03.411245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab64d7-e022-4f23-98b9-f71ebf40b636 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:03.457697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:03.458317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:03.458602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:03.458880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:03.459523 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] RBAC: Adding query filter params (type=network) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:03.459827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:03.461377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:03.470397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:03.473615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 368/721] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:03.495017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:03.650688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd753e3-7400-4b78-884d-a01b955d71cb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:03.693392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17defac7-07d4-45e0-9c66-aeb4d31e71bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:03.693941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17defac7-07d4-45e0-9c66-aeb4d31e71bc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:03.694224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17defac7-07d4-45e0-9c66-aeb4d31e71bc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:03.694505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17defac7-07d4-45e0-9c66-aeb4d31e71bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:03.705764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17defac7-07d4-45e0-9c66-aeb4d31e71bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-17defac7-07d4-45e0-9c66-aeb4d31e71bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:03.711048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:03.712037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:03.713100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 354/722] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:03.721157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:03.831040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ee81c-a3d1-48b4-b060-f873be65329e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:03.831366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ee81c-a3d1-48b4-b060-f873be65329e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:03.831598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ee81c-a3d1-48b4-b060-f873be65329e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:03.833463 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 355/723] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:03.838981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c664eb8-d4b1-44c6-a07d-99d63f7be378 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:03.839263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c664eb8-d4b1-44c6-a07d-99d63f7be378 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:03.839537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c664eb8-d4b1-44c6-a07d-99d63f7be378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:03.849414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab089b7-459b-46f5-ab7e-3ba0e03b8058 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:03.871327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7c664eb8-d4b1-44c6-a07d-99d63f7be378 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:03.875336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7c664eb8-d4b1-44c6-a07d-99d63f7be378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:03.910010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:03.911180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:03.911180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:03.911382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:03.912078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:03.912418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:03.913846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:03.928405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:03.936328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 369/724] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:03.943006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:04.019268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:04.052521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 356/725] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 13:19:04.063409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94e9f6-b8e3-4499-8056-16627ded29ad None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:04.112876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:04.113557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:04.113849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:04.114119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:04.129505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:04.131982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:04.151758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:04.153012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:04.153933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:04.153933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:04.153933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:04.154292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 370/726] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:04.160040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c664eb8-d4b1-44c6-a07d-99d63f7be378 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:04.312552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c664eb8-d4b1-44c6-a07d-99d63f7be378 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:04.367431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:04.368381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:04.368850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:04.369338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:04.370207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:04.370706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:04.372296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:04.387224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 357/727] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:04.393942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:04.528506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23805c-e761-4367-ae2c-cedf17c355fe None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:04.540985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:04.553656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:04.585645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:04.585645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:04.585645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:04.585645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:04.586057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:04.598648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:04.600152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:04.622966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:04.623970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:19:04.625002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:04.625563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 371/728] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:04.635750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:04.705611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f078e-3dcb-4be4-9e1b-756196bb336c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:04.744729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce683be-48bb-4c42-a5c0-7c0122aae488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:04.747978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce683be-48bb-4c42-a5c0-7c0122aae488 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:04.747978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce683be-48bb-4c42-a5c0-7c0122aae488 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:04.747978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce683be-48bb-4c42-a5c0-7c0122aae488 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:04.747978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce683be-48bb-4c42-a5c0-7c0122aae488 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:04.748497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce683be-48bb-4c42-a5c0-7c0122aae488 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:04.822616 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3877a6-6038-4ce7-9265-8894bd1e45e1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:04.847144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 358/729] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:19:04 2022] POST /identity/v3/services => generated 251 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:19:04.853721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb3247-eb1d-4d2b-89f7-259049612253 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:04.854003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb3247-eb1d-4d2b-89f7-259049612253 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:04.857938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb3247-eb1d-4d2b-89f7-259049612253 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:04.857938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 359/730] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:04.861671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adcf39-c80e-4a5f-98bd-bef2c09c4306 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:04.862298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adcf39-c80e-4a5f-98bd-bef2c09c4306 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:04.863109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68adcf39-c80e-4a5f-98bd-bef2c09c4306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:04.901536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-68adcf39-c80e-4a5f-98bd-bef2c09c4306 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:04.902927 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-68adcf39-c80e-4a5f-98bd-bef2c09c4306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:04.924335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:04.925409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:04.925894 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:04.926451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:04.927536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:04.928103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:04.930042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:04.974469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 372/731] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 13:19:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:04.980765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:05.012284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 360/732] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:19:05.013844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c46c0-0fcb-45d7-a6db-5688f292d91e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:05.014135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c46c0-0fcb-45d7-a6db-5688f292d91e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:05.014410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c46c0-0fcb-45d7-a6db-5688f292d91e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:05.016032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 361/733] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:05.026404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f2c1b-240f-49d0-931f-614101b3328e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:05.026710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f2c1b-240f-49d0-931f-614101b3328e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:05.026958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f2c1b-240f-49d0-931f-614101b3328e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:05.039864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:05.049179 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f13f2c1b-240f-49d0-931f-614101b3328e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:05.054675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f13f2c1b-240f-49d0-931f-614101b3328e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:05.055280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:05.112174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8816d7-a3c6-4edf-9fd1-973c119a1653 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:05.129830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 362/734] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:19:05.169386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710fdeb-6a5e-4a23-850b-c828f15b46e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:05.170004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710fdeb-6a5e-4a23-850b-c828f15b46e0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:05.170330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710fdeb-6a5e-4a23-850b-c828f15b46e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:05.170606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710fdeb-6a5e-4a23-850b-c828f15b46e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:05.260968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710fdeb-6a5e-4a23-850b-c828f15b46e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:05.262898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710fdeb-6a5e-4a23-850b-c828f15b46e0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:05.303543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:05.308558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f2c1b-240f-49d0-931f-614101b3328e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:05.441167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f2c1b-240f-49d0-931f-614101b3328e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:05.483392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 373/735] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:04 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 13:19:05.490318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710fdeb-6a5e-4a23-850b-c828f15b46e0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:05.493038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374b250-5df7-4b47-9996-80519fa5e6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:05.493966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374b250-5df7-4b47-9996-80519fa5e6ef None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:05.494455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374b250-5df7-4b47-9996-80519fa5e6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:05.494934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374b250-5df7-4b47-9996-80519fa5e6ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:05.499286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:05.513992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374b250-5df7-4b47-9996-80519fa5e6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-0374b250-5df7-4b47-9996-80519fa5e6ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:05.519962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:05.520954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 363/736] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:05.524887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374b250-5df7-4b47-9996-80519fa5e6ef None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:05.632792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710fdeb-6a5e-4a23-850b-c828f15b46e0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:05.653381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374b250-5df7-4b47-9996-80519fa5e6ef None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:05.674635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:05.686682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:05.690182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:05.690182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:05.690182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:05.694050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb531fe-b7af-45f5-aa23-fe1e177c8f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:05.694751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb531fe-b7af-45f5-aa23-fe1e177c8f64 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:05.695128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb531fe-b7af-45f5-aa23-fe1e177c8f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:05.695498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb531fe-b7af-45f5-aa23-fe1e177c8f64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:05.702760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:05.706278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:05.708488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb531fe-b7af-45f5-aa23-fe1e177c8f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ddb531fe-b7af-45f5-aa23-fe1e177c8f64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:05.713504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:05.714726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:19:05.715773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:05.717654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 364/737] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:19:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:05.718705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb531fe-b7af-45f5-aa23-fe1e177c8f64 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:05.729778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:05.730873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:05.731934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:05.731934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:05.731934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:05.734681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 374/738] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:05.736584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:05.809215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:05.871168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d559-4c2b-43b5-937f-8077dce1bc95 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:05.876484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb531fe-b7af-45f5-aa23-fe1e177c8f64 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:05.918450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:05.919162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:05.919513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:05.919860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:05.920640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:05.921012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:05.922642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:05.928288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:05.929731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:05.930201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:05.930957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:05.931980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:05.932497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:05.934004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 375/739] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:19:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:05.934273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:05.935119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:05.952924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 365/740] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:05.954043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.002568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:06.068697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e76cd-f36a-4bf0-b665-cc01279f0c46 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.069681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78591562-b498-47ad-8892-2d7917d81f60 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.090252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:06.106265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.106883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.107166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.107496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.108071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:06.108416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.109571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adac39-b4a7-4939-aad8-c53f951acb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.109755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.110210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adac39-b4a7-4939-aad8-c53f951acb40 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.111357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adac39-b4a7-4939-aad8-c53f951acb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.111867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adac39-b4a7-4939-aad8-c53f951acb40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.127531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 376/741] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:06.127727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adac39-b4a7-4939-aad8-c53f951acb40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.127864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.127975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adac39-b4a7-4939-aad8-c53f951acb40 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-47adac39-b4a7-4939-aad8-c53f951acb40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:06.143232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:06.144263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:06.145161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:06.145161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:06.145161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:06.148556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 366/742] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:06.149652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb144d-bcf8-41e6-9d95-0282aeb7ea97 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.149941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb144d-bcf8-41e6-9d95-0282aeb7ea97 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.150220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb144d-bcf8-41e6-9d95-0282aeb7ea97 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.151705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 367/743] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:06.154871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb144d-bcf8-41e6-9d95-0282aeb7ea97 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.254539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5abb69-0cc7-46a1-bd8a-2de59c5bf32e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.256747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:06.287928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb144d-bcf8-41e6-9d95-0282aeb7ea97 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.296046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.296908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.297384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.297942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.309844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.311309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.323017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355ed7c-3321-4f26-b9ca-6914509a27e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.323643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355ed7c-3321-4f26-b9ca-6914509a27e0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.323921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355ed7c-3321-4f26-b9ca-6914509a27e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.324221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355ed7c-3321-4f26-b9ca-6914509a27e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:06.329700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:06.331281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:06.332431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:06.332431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:06.332431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:06.332671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355ed7c-3321-4f26-b9ca-6914509a27e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.333061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355ed7c-3321-4f26-b9ca-6914509a27e0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.334392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 377/744] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:06.335406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-7355ed7c-3321-4f26-b9ca-6914509a27e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:06.350652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:06.351655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:06.352655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 368/745] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:06.353650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e088e42-9054-4a4f-925a-96dcd4cd17c6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.353650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e088e42-9054-4a4f-925a-96dcd4cd17c6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.353650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e088e42-9054-4a4f-925a-96dcd4cd17c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.372344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7e088e42-9054-4a4f-925a-96dcd4cd17c6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:06.373555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7e088e42-9054-4a4f-925a-96dcd4cd17c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:06.400450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:06.450513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 369/746] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 13:19:06.456595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e088e42-9054-4a4f-925a-96dcd4cd17c6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.466876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae4045-40ee-4ed0-a273-3589151e15a9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.515048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.515958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.516494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.518418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.519257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:06.519720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.521325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.535950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 378/747] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:06.537407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.556930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:06.588872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e088e42-9054-4a4f-925a-96dcd4cd17c6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.626988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.627644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.628426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.628426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.628897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:06.629198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.630598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.647351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 370/748] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:06.647351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.663436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505675ec-6dbd-4635-8153-5a839ddd6b81 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.668707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:06.702982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.703839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.704536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.704986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.705834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:06.706306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.707885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.722886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 379/749] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:06.723978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.765132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145ea20-db8c-4276-a878-050c03a08e91 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.804820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.806227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.806517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.806791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.810918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:06.816206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.817653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:06.829275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:06.836345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 371/750] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:19:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:06.837522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.847375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f00d5-e785-4d1e-adb2-eedc8d6407b1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.883653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.884398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.884665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.884831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.893021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.894477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:06.906038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:06.907020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 380/751] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:19:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:06.909371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:06.929789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:06.950926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb3b4b-bda8-4777-8a6d-2392b0e7438e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:06.995398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:06.995941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:06.996247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:06.996522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:06.997253 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=bd886328304543828dca35df3a0b7411) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:06.997600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:06.999039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:07.014056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 372/752] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:06 2022] GET /identity/v3/role_assignments?user.id=bd886328304543828dca35df3a0b7411&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:07.014056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:07.035266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150b696-0d7b-49c3-9aae-66e9f94333e3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:07.086528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c20396-7f3e-4adb-91d2-374c68d3a475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:07.087609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c20396-7f3e-4adb-91d2-374c68d3a475 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:07.088099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c20396-7f3e-4adb-91d2-374c68d3a475 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:07.088614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c20396-7f3e-4adb-91d2-374c68d3a475 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:07.089548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c20396-7f3e-4adb-91d2-374c68d3a475 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:07.091394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c20396-7f3e-4adb-91d2-374c68d3a475 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:07.096437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:07.160029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25932c45-537d-4c0e-aa29-87534d777019 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:07.196856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:07.197469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:07.197882 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:07.198050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:07.198753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:07.199059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:07.200519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:07.220510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 373/753] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 13:19:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:07.222111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:07.225641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:07.245919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 381/754] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:19:06 2022] POST /identity/v3/services => generated 240 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:07.247584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40984b-3904-4868-b628-2f71f0e4549e None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:07.248044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40984b-3904-4868-b628-2f71f0e4549e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:07.248524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40984b-3904-4868-b628-2f71f0e4549e None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:07.258405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 382/755] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:07 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:07.260676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40984b-3904-4868-b628-2f71f0e4549e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:07.373360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd3ad1-3c39-4907-b9ae-fcfc90094556 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:07.426974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb86a8b-6e62-40d9-8a60-8361dbec0a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:07.427592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb86a8b-6e62-40d9-8a60-8361dbec0a63 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:07.427874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb86a8b-6e62-40d9-8a60-8361dbec0a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:07.428145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb86a8b-6e62-40d9-8a60-8361dbec0a63 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:07.428924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb86a8b-6e62-40d9-8a60-8361dbec0a63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:07.429229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb86a8b-6e62-40d9-8a60-8361dbec0a63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:07.429635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40984b-3904-4868-b628-2f71f0e4549e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:07.431712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb86a8b-6e62-40d9-8a60-8361dbec0a63 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:07.454937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 374/756] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:19:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:07.456232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca561b-d043-4e18-8b01-72557edd2945 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:07.456538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca561b-d043-4e18-8b01-72557edd2945 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:07.456879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca561b-d043-4e18-8b01-72557edd2945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:07.477674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-beca561b-d043-4e18-8b01-72557edd2945 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:07.479159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2c8a6-086e-4545-95e8-099765ec82b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:07.479566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-beca561b-d043-4e18-8b01-72557edd2945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:07.479700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2c8a6-086e-4545-95e8-099765ec82b9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:07.479700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2c8a6-086e-4545-95e8-099765ec82b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:07.479700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2c8a6-086e-4545-95e8-099765ec82b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:07.556406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:07.577454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 375/757] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:07.578375 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca561b-d043-4e18-8b01-72557edd2945 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:07.707326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:07.719033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2c8a6-086e-4545-95e8-099765ec82b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:07.720956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2c8a6-086e-4545-95e8-099765ec82b9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:07.761716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca561b-d043-4e18-8b01-72557edd2945 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:07.803185 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:07.803926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:07.804240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:07.804413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:07.818232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:07.819705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:07.836566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:07.837570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:07.838525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:07.838525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:07.838525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:07.838525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 376/758] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:07.849475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:07.903580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:07.909038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 383/759] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:19:07.909992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2c8a6-086e-4545-95e8-099765ec82b9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:07.979888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb9b47-d949-49cb-83f0-87c7213c6cc6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.011011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:08.019190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2c8a6-086e-4545-95e8-099765ec82b9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.023427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.023427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.023427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.023427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.023755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:08.023755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.023888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.041958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 377/760] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:08.042933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:08.056645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.057278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.057557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.057827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.058424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.059798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:08.069731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:08.070845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:08.071691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:08.077159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 384/761] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 13:19:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:08.077159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:08.228358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:08.233354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd734f6a-edb6-4ae1-b8c5-d2fca76dcc26 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.234058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2dfa7-6ad5-4b28-b8f0-5d409c3f0154 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.279614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.280313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.280593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.280878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.284640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.285276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.285550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.285815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.291440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.292939 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.301804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.303704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:08.308327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:08.309238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:08.310134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 385/762] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:08.310621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:08.326473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:08.327526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:08.328477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:08.328477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:08.328477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:08.335129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 378/763] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:08.335129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:08.343074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:08.449781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355e5c8-04fc-4c09-ae5c-a8b54511faec None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.473972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f07c02-bb43-4619-a6ae-eb558b5b5fd7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.497031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.497556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.497914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.498194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.499202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:08.499504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.501002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.508264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 386/764] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 13:19:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:08.509449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:08.529401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.530705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.531817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.532486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.534293 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:08.534825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.537285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.555397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 379/765] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:08.557624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:08.573757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:08.634206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:08.659362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85307c2-f3b2-4bd8-a114-670da50dcb4e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.699231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb446f7-5a3a-475a-b2e7-c9240042646f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.702726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.703203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.703492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.703763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.705042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:08.705348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.706958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.726189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 387/766] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:08.727540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:08.752881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.754055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.754534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.755258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.756666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:08.757467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.759921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.773362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 380/767] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 13:19:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:08.773362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:08.868800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:08.880790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd04ba-7874-410a-aec3-dac0365f853f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.899900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633b19a-900a-4391-ad63-184396a44411 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:08.920883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fda17-b673-4d5c-a034-ef3d622b688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.922178 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fda17-b673-4d5c-a034-ef3d622b688d None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.922849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fda17-b673-4d5c-a034-ef3d622b688d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.923950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fda17-b673-4d5c-a034-ef3d622b688d None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/40561a306123461d92bc669be3962db3/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.926326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fda17-b673-4d5c-a034-ef3d622b688d None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=40561a306123461d92bc669be3962db3, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.928058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fda17-b673-4d5c-a034-ef3d622b688d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.944664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:08.947313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:08.947313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:08.947313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:08.953560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:08.955761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:08.968353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:08.969576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 381/768] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:08.971218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:09.020941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 388/769] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:19:08 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/40561a306123461d92bc669be3962db3/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:19:09.022460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fda17-b673-4d5c-a034-ef3d622b688d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:09.026603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:09.147124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38023bc-cd45-4a9b-bdeb-8c7a753d8c00 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:09.212872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a773e-af55-4598-9f45-16bf90504005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:09.213498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a773e-af55-4598-9f45-16bf90504005 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.213779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a773e-af55-4598-9f45-16bf90504005 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.214058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a773e-af55-4598-9f45-16bf90504005 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.214771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a773e-af55-4598-9f45-16bf90504005 None admin] RBAC: Adding query filter params (interface=public, service_id=964bad2346b44cac946732e23ab80977, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:09.215083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a773e-af55-4598-9f45-16bf90504005 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:09.216515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555a773e-af55-4598-9f45-16bf90504005 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:09.218461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:09.227878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 382/770] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 13:19:08 2022] GET /identity/v3/endpoints?service_id=964bad2346b44cac946732e23ab80977&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:09.229025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929db95b-ada1-4552-8817-fdd0631bf2cf None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.233957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929db95b-ada1-4552-8817-fdd0631bf2cf None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.233957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929db95b-ada1-4552-8817-fdd0631bf2cf None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.233957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 383/771] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:09.236552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad71d-7741-43fa-9641-f39236fffc00 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.236801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad71d-7741-43fa-9641-f39236fffc00 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.237083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad71d-7741-43fa-9641-f39236fffc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.256945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fda17-b673-4d5c-a034-ef3d622b688d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:09.259071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fe7ad71d-7741-43fa-9641-f39236fffc00 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:09.260556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fe7ad71d-7741-43fa-9641-f39236fffc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:09.301018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:09.302719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.303393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.304571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.305582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:09.305990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:09.308215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:09.324877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 389/772] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:09.325500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:09.358429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 384/773] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:09.359404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ad71d-7741-43fa-9641-f39236fffc00 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:09.420660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:09.473690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541fce4-1997-4f36-aaf8-8666ac448cf1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:09.513633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d62f16-f8f3-4df3-931f-fdd418bfeaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:09.514643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d62f16-f8f3-4df3-931f-fdd418bfeaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.515387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d62f16-f8f3-4df3-931f-fdd418bfeaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.515872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d62f16-f8f3-4df3-931f-fdd418bfeaa4 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.518017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d62f16-f8f3-4df3-931f-fdd418bfeaa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:09.519569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d62f16-f8f3-4df3-931f-fdd418bfeaa4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:09.525910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 390/774] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:19:09 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:09.527426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693ff0f-cb45-4686-a1d2-05f218b42276 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.527934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693ff0f-cb45-4686-a1d2-05f218b42276 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.528521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693ff0f-cb45-4686-a1d2-05f218b42276 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.535463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 391/775] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:09.537627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dba26-fb31-451b-9bc8-1561de44549b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.538154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dba26-fb31-451b-9bc8-1561de44549b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.538653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dba26-fb31-451b-9bc8-1561de44549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.558426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ad71d-7741-43fa-9641-f39236fffc00 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:09.559534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ab0dba26-fb31-451b-9bc8-1561de44549b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:09.560792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ab0dba26-fb31-451b-9bc8-1561de44549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:09.628620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:09.648638 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:09.649282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.649567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.649842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.650595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=b56fb09428cb4ab2b49ca199058a5a7a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:09.650906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:09.652437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:09.670699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 385/776] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:09 2022] GET /identity/v3/role_assignments?user.id=b56fb09428cb4ab2b49ca199058a5a7a&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:09.699222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 392/777] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:09.699420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:09.819177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20385958-677a-46d5-af06-9a20c7052cc5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:09.865067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254d786-549a-42cc-834b-2540080c7d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:09.865722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254d786-549a-42cc-834b-2540080c7d1b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:09.866009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254d786-549a-42cc-834b-2540080c7d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:09.866283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254d786-549a-42cc-834b-2540080c7d1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:09.881823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254d786-549a-42cc-834b-2540080c7d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1254d786-549a-42cc-834b-2540080c7d1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:09.887166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:09.888116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:09.888987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 386/778] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:09.894410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dba26-fb31-451b-9bc8-1561de44549b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:09.980124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:10.034751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dba26-fb31-451b-9bc8-1561de44549b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:10.050189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254d786-549a-42cc-834b-2540080c7d1b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:10.103660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:10.105534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:10.106070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:10.107454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:10.108392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:10.109082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:10.111993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:10.131200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 393/779] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:10.143347 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:10.168598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:10.247332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254d786-549a-42cc-834b-2540080c7d1b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:10.284664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aadd4-21dc-4618-9876-6f5a4b3df7d0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:10.312794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:10.313905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:10.314569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:10.314856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:10.315136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:10.315740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:10.317159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:10.327961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:10.328642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:10.329087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:10.330176 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:10.343597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:10.346131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:10.347105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:10.347940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:10.348704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:10.348786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 387/780] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:10 2022] GET /identity/v3/services/network => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:10.357276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:10.369256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:10.371478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:10.374757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:10.374757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:10.374757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:10.374757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 394/781] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:10.378740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:10.494912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5274c25-1376-4c00-98d1-847a3d898139 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:10.535791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:10.536552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:10.536912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:10.537360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:10.538093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:10.538472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:10.539936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:10.554818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 395/782] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:10.560875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:10.616666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:10.665326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f6f31-5ac7-4a5a-b5c4-5a9c2fa86ed2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:10.700496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:10.721850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:10.722468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:10.722778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:10.723028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:10.723676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] RBAC: Adding query filter params (name=network) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:10.724050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:10.725604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:10.730986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 388/783] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:10.743631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:10.761550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d229609-e252-4231-ac55-68d618513c1b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:10.862485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:10.863126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:10.863404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:10.863673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:10.874862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:10.876331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:10.897759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:10.898752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:10.899685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 396/784] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:10.905490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:11.027322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609966a7-f112-4157-89ba-a007d8b25043 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:11.071203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:11.083431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efdd16-db94-43f7-917f-38b293368da6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:11.095207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:11.096834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.103524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.104394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.105923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] RBAC: Adding query filter params (type=network) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:11.106754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:11.109763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:11.170049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:11.171465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.172328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.173382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.175423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:11.176379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:11.179366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:11.182109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 389/785] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:11.188967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:11.217174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 397/786] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 13:19:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:11.223829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:11.291741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:11.379855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfdf7f-3e92-4166-a0c5-d7916f095d6f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:11.407949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fb352-5e4b-465b-a846-bcf7847086dd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:11.420139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0b89d-6ffd-428d-accd-cd717ab6554a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:11.420717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0b89d-6ffd-428d-accd-cd717ab6554a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.421077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0b89d-6ffd-428d-accd-cd717ab6554a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.421354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0b89d-6ffd-428d-accd-cd717ab6554a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.422211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0b89d-6ffd-428d-accd-cd717ab6554a None admin] RBAC: Adding query filter params (interface=public, service_id=d87258d2558542899365865c9146b473, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:11.422524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0b89d-6ffd-428d-accd-cd717ab6554a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:11.424828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0b89d-6ffd-428d-accd-cd717ab6554a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:11.431326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 390/787] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 13:19:11 2022] GET /identity/v3/endpoints?service_id=d87258d2558542899365865c9146b473&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:11.433525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3de517-2b53-4334-84d7-27198a37855e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.434353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3de517-2b53-4334-84d7-27198a37855e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.434637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3de517-2b53-4334-84d7-27198a37855e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.442081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 391/788] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:11.443110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3de517-2b53-4334-84d7-27198a37855e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:11.457304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98dbbc-ec4d-43a0-8d34-dcb950772dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:11.458691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98dbbc-ec4d-43a0-8d34-dcb950772dad None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.459503 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98dbbc-ec4d-43a0-8d34-dcb950772dad None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.460007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98dbbc-ec4d-43a0-8d34-dcb950772dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.473621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98dbbc-ec4d-43a0-8d34-dcb950772dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:11.476586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98dbbc-ec4d-43a0-8d34-dcb950772dad None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:11.490300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7b98dbbc-ec4d-43a0-8d34-dcb950772dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:11.498479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:11.506689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:11.507663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:11.507663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:11.507663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:11.507663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 398/789] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:11.507663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9afa9c-5d62-4757-8da3-de7e54cee9a2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.507663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9afa9c-5d62-4757-8da3-de7e54cee9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.507663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9afa9c-5d62-4757-8da3-de7e54cee9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.526997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fe9afa9c-5d62-4757-8da3-de7e54cee9a2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:11.528668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fe9afa9c-5d62-4757-8da3-de7e54cee9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:11.582452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3de517-2b53-4334-84d7-27198a37855e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:11.594639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:11.640214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 399/790] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:11.640214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9afa9c-5d62-4757-8da3-de7e54cee9a2 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:11.649348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9ac71-f71f-4971-adb4-2c24cc1bb5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:11.649974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ac71-f71f-4971-adb4-2c24cc1bb5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.650264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ac71-f71f-4971-adb4-2c24cc1bb5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.650536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ac71-f71f-4971-adb4-2c24cc1bb5f1 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.651196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9ac71-f71f-4971-adb4-2c24cc1bb5f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:11.652589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9ac71-f71f-4971-adb4-2c24cc1bb5f1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:11.661638 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 392/791] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:19:11 2022] GET /identity/v3/services => generated 772 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:11.765422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9afa9c-5d62-4757-8da3-de7e54cee9a2 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:11.801992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:11.808289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7831c78-7636-46d5-9cc8-b1517d2908e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:11.808918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831c78-7636-46d5-9cc8-b1517d2908e8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.809201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831c78-7636-46d5-9cc8-b1517d2908e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.809478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7831c78-7636-46d5-9cc8-b1517d2908e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.810139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7831c78-7636-46d5-9cc8-b1517d2908e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:11.810446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7831c78-7636-46d5-9cc8-b1517d2908e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:11.811832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7831c78-7636-46d5-9cc8-b1517d2908e8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:11.825800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 400/792] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:11.832091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9ac71-f71f-4971-adb4-2c24cc1bb5f1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:11.887100 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d24cb-3523-42a6-b3d0-5737500731c7 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.887372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d24cb-3523-42a6-b3d0-5737500731c7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.887655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d24cb-3523-42a6-b3d0-5737500731c7 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.889349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 401/793] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:11.897338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b472880-00d9-4b91-a1e0-1742e2eb05fe None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:11.897775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b472880-00d9-4b91-a1e0-1742e2eb05fe None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:11.898161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b472880-00d9-4b91-a1e0-1742e2eb05fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:11.921625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2b472880-00d9-4b91-a1e0-1742e2eb05fe None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:11.924024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2b472880-00d9-4b91-a1e0-1742e2eb05fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:12.014745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 402/794] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:12.024882 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9ac71-f71f-4971-adb4-2c24cc1bb5f1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:12.068975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73990d48-2d28-4bc7-aaea-c558159fa855 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.069241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73990d48-2d28-4bc7-aaea-c558159fa855 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.069469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73990d48-2d28-4bc7-aaea-c558159fa855 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.071117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 403/795] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:12.075098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73990d48-2d28-4bc7-aaea-c558159fa855 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:12.078242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498b551-9909-45a0-937e-32ec4a4bffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:12.079164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b551-9909-45a0-937e-32ec4a4bffdf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.079511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b551-9909-45a0-937e-32ec4a4bffdf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.079772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b551-9909-45a0-937e-32ec4a4bffdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.088543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498b551-9909-45a0-937e-32ec4a4bffdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:12.090391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498b551-9909-45a0-937e-32ec4a4bffdf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-8498b551-9909-45a0-937e-32ec4a4bffdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:12.103760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:12.104770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:12.105669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:12.105669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:12.105669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:12.105669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:12.105669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:12.105669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:12.105669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:12.105669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:12.107559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 393/796] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:12.109016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423d500-0b7c-4fd7-97b4-e8bc6b545488 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.109307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423d500-0b7c-4fd7-97b4-e8bc6b545488 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.109591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423d500-0b7c-4fd7-97b4-e8bc6b545488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.129124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a423d500-0b7c-4fd7-97b4-e8bc6b545488 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:12.130442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a423d500-0b7c-4fd7-97b4-e8bc6b545488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:12.161897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:12.205390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73990d48-2d28-4bc7-aaea-c558159fa855 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:12.228165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 394/797] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:12.229497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423d500-0b7c-4fd7-97b4-e8bc6b545488 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:12.234110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:12.264541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bf5c4-e4cb-495d-94b9-995b274f1268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:12.265861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bf5c4-e4cb-495d-94b9-995b274f1268 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.266277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bf5c4-e4cb-495d-94b9-995b274f1268 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.268291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bf5c4-e4cb-495d-94b9-995b274f1268 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.287606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bf5c4-e4cb-495d-94b9-995b274f1268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-cc6bf5c4-e4cb-495d-94b9-995b274f1268 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:12.294172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:12.295147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:12.296034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 404/798] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:12.302421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bf5c4-e4cb-495d-94b9-995b274f1268 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:12.364144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423d500-0b7c-4fd7-97b4-e8bc6b545488 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:12.405167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:12.405889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.406238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.406584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.407671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:12.408076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:12.409644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:12.424137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 395/799] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:12.425337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:12.442334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bf5c4-e4cb-495d-94b9-995b274f1268 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:12.456767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:12.497597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:12.499392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.499880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.500372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.501643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:12.502188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:12.504563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:12.518568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 405/800] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:12.519655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:12.564457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bfdd-52ee-402b-b336-b78fa8d8ece2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:12.579979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:12.605905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a85ae1-fbdf-4069-bfc7-f33066d5a29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:12.606550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a85ae1-fbdf-4069-bfc7-f33066d5a29b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.606839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a85ae1-fbdf-4069-bfc7-f33066d5a29b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.607180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a85ae1-fbdf-4069-bfc7-f33066d5a29b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.619658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a85ae1-fbdf-4069-bfc7-f33066d5a29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e4a85ae1-fbdf-4069-bfc7-f33066d5a29b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:12.625040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:12.626255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:12.627256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 396/801] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:19:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:12.628031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a85ae1-fbdf-4069-bfc7-f33066d5a29b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:12.663344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c5579-d5a9-4f8e-ac94-35f85c81045e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:12.704628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a62c28-eb5c-4620-a1c6-8519e9294e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:12.705303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a62c28-eb5c-4620-a1c6-8519e9294e39 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.705962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a62c28-eb5c-4620-a1c6-8519e9294e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.706253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a62c28-eb5c-4620-a1c6-8519e9294e39 None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/bd886328304543828dca35df3a0b7411/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.708466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a62c28-eb5c-4620-a1c6-8519e9294e39 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=bd886328304543828dca35df3a0b7411, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:12.709949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a62c28-eb5c-4620-a1c6-8519e9294e39 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:12.749070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:12.768504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a85ae1-fbdf-4069-bfc7-f33066d5a29b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:12.787199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 406/802] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:19:12 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/bd886328304543828dca35df3a0b7411/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:19:12.788300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a62c28-eb5c-4620-a1c6-8519e9294e39 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:12.819609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:12.820683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:12.820975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:12.821295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:12.840522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:12.842732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:12.862586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:12.863608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:12.864547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:12.864547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:12.864547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:12.868370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 397/803] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:12.870618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:12.960591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a62c28-eb5c-4620-a1c6-8519e9294e39 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:12.962804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:13.005420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.006214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.006550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.007146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.008136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.010067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.010588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fe6b8-f204-4e34-b89a-0d17c6189e20 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:13.024122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:13.025193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 13:19:13.026009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:13.027034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 407/804] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 13:19:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:13.027034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.054848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.055668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.055950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.056253 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.057095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:13.057398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.059433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.071947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 398/805] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:19:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:13.072870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.073552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:13.151412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6fca-6db9-4cd1-9a36-15519679f30f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.189390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.190397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.190871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.191409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.192403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:13.192960 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.194625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.209505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 408/806] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:13.209695 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f089d6-e86f-4586-b627-01a5964d330e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.211499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.244636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:13.260880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.261497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.261783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.262049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.262684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:13.262981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.264996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.277207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 399/807] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 13:19:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:13.277207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.345546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:13.361885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35318f-be46-4be6-aed1-b5484ffc3420 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.394373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3a192-313d-47e9-9a68-d27a15d35d2a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.407755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.409338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.409774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.410195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.423843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.426046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.435422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.436123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.436584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.437037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:13.445251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:13.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:13.447532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:13.447532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:13.447532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:13.449001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.449741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 409/808] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:13.450687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.455763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:13.469045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:13.470334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:13.471357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:13.476328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 400/809] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:19:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:13.477330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.553229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:13.582553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dbe61-57a7-461f-a7a5-6944837e523c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.599935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb645c-0dde-406f-85b9-eae837809b0a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.621394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:13.628686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.630428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.630703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.630975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.631605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:13.635561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.635561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.639313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 410/810] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 13:19:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:13.644253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.647027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.647639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.647962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.648280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.648930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:13.649230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.650716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.668238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 401/811] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:13.668238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.805938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257bcfb-91fa-41f1-85e3-4da37ea42b2d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.817588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d2e05-95e7-43b9-9be6-4c8ded6236e2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:13.850891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.851397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.851685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.851961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.852899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:13.853078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.854554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.867022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b9213-9127-478e-a3b7-0d60bd9fa3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:13.868020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b9213-9127-478e-a3b7-0d60bd9fa3ca None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:13.868595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b9213-9127-478e-a3b7-0d60bd9fa3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:13.869112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b9213-9127-478e-a3b7-0d60bd9fa3ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:13.869973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b9213-9127-478e-a3b7-0d60bd9fa3ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:13.872297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b9213-9127-478e-a3b7-0d60bd9fa3ca None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:13.876573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:13.880012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 411/812] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:19:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:13.881062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:13.999141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e4b84-1d80-495e-9e37-d3e3e7e79c45 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:14.044455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 402/813] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:19:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:14.045637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b9213-9127-478e-a3b7-0d60bd9fa3ca None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:14.046212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9f538-69c6-4520-9323-feeb7bc96b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:14.047141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9f538-69c6-4520-9323-feeb7bc96b3e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.048540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9f538-69c6-4520-9323-feeb7bc96b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.049107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9f538-69c6-4520-9323-feeb7bc96b3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.068060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9f538-69c6-4520-9323-feeb7bc96b3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:14.070112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9f538-69c6-4520-9323-feeb7bc96b3e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-91c9f538-69c6-4520-9323-feeb7bc96b3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:14.092596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:14.093608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:14.095129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:14.098502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 412/814] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:19:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:14.100101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a663b76-9e60-4cf4-8d10-898b9a3aef2b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.100544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a663b76-9e60-4cf4-8d10-898b9a3aef2b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.101060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a663b76-9e60-4cf4-8d10-898b9a3aef2b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.103619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 413/815] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:14.104553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a663b76-9e60-4cf4-8d10-898b9a3aef2b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:14.124350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:14.196149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:14.208210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b9213-9127-478e-a3b7-0d60bd9fa3ca None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:14.255717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27ab66-779d-4a58-a3b0-d14b1d6f9dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:14.256906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ab66-779d-4a58-a3b0-d14b1d6f9dad None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.257192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ab66-779d-4a58-a3b0-d14b1d6f9dad None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.257472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ab66-779d-4a58-a3b0-d14b1d6f9dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.261715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a663b76-9e60-4cf4-8d10-898b9a3aef2b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:14.312460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164d5fd-f0e0-498f-8ad0-3e15e8af5cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:14.313632 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164d5fd-f0e0-498f-8ad0-3e15e8af5cee None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.313929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164d5fd-f0e0-498f-8ad0-3e15e8af5cee None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.315142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164d5fd-f0e0-498f-8ad0-3e15e8af5cee None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.315656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164d5fd-f0e0-498f-8ad0-3e15e8af5cee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:14.315970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164d5fd-f0e0-498f-8ad0-3e15e8af5cee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:14.317462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164d5fd-f0e0-498f-8ad0-3e15e8af5cee None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:14.343502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 414/816] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 13:19:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:14.346370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16aeef-2b10-4d54-931d-c944be95bd35 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.346599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16aeef-2b10-4d54-931d-c944be95bd35 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.346868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16aeef-2b10-4d54-931d-c944be95bd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.355866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27ab66-779d-4a58-a3b0-d14b1d6f9dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:14.357786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27ab66-779d-4a58-a3b0-d14b1d6f9dad None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:14.367792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7c16aeef-2b10-4d54-931d-c944be95bd35 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:14.369419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7c16aeef-2b10-4d54-931d-c944be95bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:14.410170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:14.469212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 415/817] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:14.469212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16aeef-2b10-4d54-931d-c944be95bd35 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:14.533095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 403/818] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:19:14.534036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27ab66-779d-4a58-a3b0-d14b1d6f9dad None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:14.550397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:14.628079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16aeef-2b10-4d54-931d-c944be95bd35 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:14.690980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:14.692551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.692832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.693102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.720337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:14.721430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27ab66-779d-4a58-a3b0-d14b1d6f9dad None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:14.724422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:14.740563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:14.741608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:14.742511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:14.742511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:14.742511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:14.742511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 416/819] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:14.753222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:14.785964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:14.786684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.786969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.787240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.791113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:14.800201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:14.812436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:14.818460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:14.819654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:14.820703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:14.820703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:14.820703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:14.824283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 404/820] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:14.825183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:14.884823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fe702-8352-4a19-aa22-d6b96e7cbc80 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:14.888384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:14.924979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:14.926053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.926543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.927050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.927918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:14.928564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:14.930346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:14.939734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80225cd2-574e-4816-85ac-5928287800c7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:14.946280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 417/821] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:14.947126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:14.977403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b518-d431-4af4-a4cb-53a834d06644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:14.978037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b518-d431-4af4-a4cb-53a834d06644 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.978321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b518-d431-4af4-a4cb-53a834d06644 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.978599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b518-d431-4af4-a4cb-53a834d06644 None admin] PATH_INFO: `/v3/services/network` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.979186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679b518-d431-4af4-a4cb-53a834d06644 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:14.980591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679b518-d431-4af4-a4cb-53a834d06644 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d679b518-d431-4af4-a4cb-53a834d06644 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:14.989822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:14.990864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:14.991697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 405/822] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:14 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:14.992523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f11e-d6be-4cca-9479-75d13ef6e92e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:14.992523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f11e-d6be-4cca-9479-75d13ef6e92e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:14.993203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f11e-d6be-4cca-9479-75d13ef6e92e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:14.994243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 406/823] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:14.994893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f11e-d6be-4cca-9479-75d13ef6e92e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:15.069491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:15.084958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c6312-1aa2-4e5b-8beb-19c494652cd2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:15.128943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:15.129762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.130253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.130999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.131481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:15.132029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:15.132616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f11e-d6be-4cca-9479-75d13ef6e92e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:15.134480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:15.154876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 418/824] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:15.154876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:15.177103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:15.191505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38012a8-4f33-4794-a45c-bc53c979ffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:15.192137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38012a8-4f33-4794-a45c-bc53c979ffb7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.192469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38012a8-4f33-4794-a45c-bc53c979ffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.192754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38012a8-4f33-4794-a45c-bc53c979ffb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.201187 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38012a8-4f33-4794-a45c-bc53c979ffb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:15.202733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38012a8-4f33-4794-a45c-bc53c979ffb7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d38012a8-4f33-4794-a45c-bc53c979ffb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:15.215521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:15.216727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:15.217613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 407/825] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:15.218222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6170a2e-9524-4836-9e40-5bb20ec978b7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.218469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6170a2e-9524-4836-9e40-5bb20ec978b7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.218742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6170a2e-9524-4836-9e40-5bb20ec978b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.238421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f6170a2e-9524-4836-9e40-5bb20ec978b7 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:15.239636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f6170a2e-9524-4836-9e40-5bb20ec978b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:15.287879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a900365-38ef-4d44-981a-485e3241a70c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:15.327379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 408/826] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:15.328379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6170a2e-9524-4836-9e40-5bb20ec978b7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:15.345702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:15.346634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.347064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.347465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.348665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] RBAC: Adding query filter params (name=network) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:15.349327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:15.352065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:15.363324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 419/827] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:15.364420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:15.398710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:15.467109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:15.488752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafad269-74bf-41e1-878f-f9256d844ece None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:15.498036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6170a2e-9524-4836-9e40-5bb20ec978b7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:15.529972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:15.530729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.531058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.531404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.533019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:15.533396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:15.534910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:15.548862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 420/828] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:15.553168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:15.567024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:15.569661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.570807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.571570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.581623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:15.583181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:15.595877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:15.597679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 409/829] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:15.599441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:15.697908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3d89c-ca49-4a76-b56b-ce1ba38fca46 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:15.736970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:15.737647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.737879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.738155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.738926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] RBAC: Adding query filter params (type=network) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:15.739104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:15.740625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:15.754491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 421/830] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:15.755158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:15.757002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fae98-5105-48a2-a4dc-182bdd4019c1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:15.758764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:15.759839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:15.801315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e20d96-eace-4341-96a5-dc1959f279e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:15.801996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e20d96-eace-4341-96a5-dc1959f279e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.802281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e20d96-eace-4341-96a5-dc1959f279e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.802612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e20d96-eace-4341-96a5-dc1959f279e2 None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/b56fb09428cb4ab2b49ca199058a5a7a/roles/8760c26505544fdda1848788d4df9b5a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.818811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e20d96-eace-4341-96a5-dc1959f279e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=b56fb09428cb4ab2b49ca199058a5a7a, role_id=8760c26505544fdda1848788d4df9b5a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:15.820289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e20d96-eace-4341-96a5-dc1959f279e2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:15.872449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f9de-c932-4331-a54b-2c782397081d None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:15.910166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:15.910790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:15.911195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:15.911517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:15.912162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:15.912506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:15.914146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:15.927877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 410/831] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:19:15 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/b56fb09428cb4ab2b49ca199058a5a7a/roles/8760c26505544fdda1848788d4df9b5a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:19:15.928988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e20d96-eace-4341-96a5-dc1959f279e2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:15.933217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 422/832] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:15.936345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:16.019091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:16.065567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a736fae-fb75-4253-82e1-946d3b0048bb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:16.074984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e20d96-eace-4341-96a5-dc1959f279e2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:16.117966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffd8d7-47f6-46cd-aa83-f89ef826c93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:16.118589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffd8d7-47f6-46cd-aa83-f89ef826c93a None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:16.118868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffd8d7-47f6-46cd-aa83-f89ef826c93a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:16.119138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffd8d7-47f6-46cd-aa83-f89ef826c93a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:16.119847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffd8d7-47f6-46cd-aa83-f89ef826c93a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:16.121267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffd8d7-47f6-46cd-aa83-f89ef826c93a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:16.122704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec59ac-04d6-4c4e-9cac-a2e44fd2e9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:16.123350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec59ac-04d6-4c4e-9cac-a2e44fd2e9de None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:16.123630 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec59ac-04d6-4c4e-9cac-a2e44fd2e9de None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:16.124056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec59ac-04d6-4c4e-9cac-a2e44fd2e9de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:16.136962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec59ac-04d6-4c4e-9cac-a2e44fd2e9de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-25ec59ac-04d6-4c4e-9cac-a2e44fd2e9de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:16.141832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:16.149266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 411/833] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:16.150122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec59ac-04d6-4c4e-9cac-a2e44fd2e9de None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:16.161983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:16.212337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 423/834] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:19:15 2022] POST /identity/v3/endpoints => generated 337 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:16.213575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffd8d7-47f6-46cd-aa83-f89ef826c93a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:16.272649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec59ac-04d6-4c4e-9cac-a2e44fd2e9de None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:16.334052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed5dc3-c862-4b68-9606-8ac201f3c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:16.334852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5dc3-c862-4b68-9606-8ac201f3c31e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:16.335035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5dc3-c862-4b68-9606-8ac201f3c31e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:16.335337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed5dc3-c862-4b68-9606-8ac201f3c31e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:16.336121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed5dc3-c862-4b68-9606-8ac201f3c31e None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=40561a306123461d92bc669be3962db3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:16.336457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed5dc3-c862-4b68-9606-8ac201f3c31e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:16.337968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed5dc3-c862-4b68-9606-8ac201f3c31e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:16.345363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 412/835] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:16 2022] GET /identity/v3/role_assignments?user.id=40561a306123461d92bc669be3962db3&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:16.346767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b972a-0a7c-476e-b035-866403c388e9 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:16.347064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b972a-0a7c-476e-b035-866403c388e9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:16.347343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b972a-0a7c-476e-b035-866403c388e9 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:16.352944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 413/836] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:16.354923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ed8f5-1139-4bb4-853c-1ec211e3b209 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:16.355212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ed8f5-1139-4bb4-853c-1ec211e3b209 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:16.355495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ed8f5-1139-4bb4-853c-1ec211e3b209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:16.373498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffd8d7-47f6-46cd-aa83-f89ef826c93a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:16.377147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b40ed8f5-1139-4bb4-853c-1ec211e3b209 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:16.378224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b40ed8f5-1139-4bb4-853c-1ec211e3b209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:16.399730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:16.418845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:16.419936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:16.420428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:16.420947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:16.421941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:16.423018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:16.425116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:16.437987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 424/837] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:16.444056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:16.504356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 414/838] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:16.573985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c0787-6a49-45d8-b036-39efb63d719b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:16.611892 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:16.612937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:16.613414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:16.613899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:16.628020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:16.630044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:16.646721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:16.647725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:16.648700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:16.648700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:16.648700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:16.650624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 425/839] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:16.657932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ed8f5-1139-4bb4-853c-1ec211e3b209 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:16.660540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:16.784563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ed8f5-1139-4bb4-853c-1ec211e3b209 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:16.830740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:16.831358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:16.831636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:16.831907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:16.832602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:16.832909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:16.834302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:16.850102 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 415/840] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:16.855376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:16.860714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:16.898003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:16.952262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:16.984415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476e864-e43e-4fcb-a3e6-fcf36247907b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:17.019257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09269cc8-6848-41bf-b88f-81a48e4f8db5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:17.035546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:17.036208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:17.036493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:17.037033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:17.047146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:17.049731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:17.066169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:17.071830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 13:19:17.072861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:17.073522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:17.073522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:17.073522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:17.073731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 426/841] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:17.073800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:17.073800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:17.073800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:17.074641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:17.082054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:17.083262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:17.084309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:17.091231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 416/842] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:17.092472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:17.206708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:17.214276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9241e0-197e-4ddc-9fa6-819bb123ef68 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:17.235552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c8304-2268-4d22-87f2-da9d2cfbbc84 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:17.255040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:17.255806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:17.256204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:17.256620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:17.257518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:17.257946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:17.259474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:17.269043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:17.281022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:17.282538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:17.283041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:17.283450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:17.284341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:17.284804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:17.286360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:17.287754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 427/843] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 13:19:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:17.289136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:17.294148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 417/844] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:17.301543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:17.437267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025823b-e5d7-4dac-bdbd-af56ee15f823 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:17.449222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd0ca0-3457-4413-99c4-56fdd35a9da5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:17.482328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7e6cc-7c55-43a0-9aff-4bab8df09ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:17.483060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7e6cc-7c55-43a0-9aff-4bab8df09ddb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:17.483460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7e6cc-7c55-43a0-9aff-4bab8df09ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:17.483840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7e6cc-7c55-43a0-9aff-4bab8df09ddb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:17.497303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:17.498574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:17.499182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:17.499732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:17.500740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:17.501228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:17.503525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:17.510172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:17.511318 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 418/845] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:17.576009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7e6cc-7c55-43a0-9aff-4bab8df09ddb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:17.577880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7e6cc-7c55-43a0-9aff-4bab8df09ddb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:17.699652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 428/846] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:19:17.707140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:17.778827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:17.830599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda6621-6afb-4635-9591-1fd697e4698f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:17.867670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f01840-4d9e-48da-a4ff-2aa2dabfff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:17.868346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f01840-4d9e-48da-a4ff-2aa2dabfff8a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:17.868633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f01840-4d9e-48da-a4ff-2aa2dabfff8a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:17.868921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f01840-4d9e-48da-a4ff-2aa2dabfff8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:17.881290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f01840-4d9e-48da-a4ff-2aa2dabfff8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:17.882753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f01840-4d9e-48da-a4ff-2aa2dabfff8a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-84f01840-4d9e-48da-a4ff-2aa2dabfff8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:17.898439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:17.899450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:17.900771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:17.900771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:17.900771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:17.900771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 419/847] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:17.905658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7e6cc-7c55-43a0-9aff-4bab8df09ddb None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:17.966245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:17.998585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc5480-b1cd-49b6-9591-86ab6bbb473c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:17.998918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc5480-b1cd-49b6-9591-86ab6bbb473c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:17.999139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc5480-b1cd-49b6-9591-86ab6bbb473c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.000826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 420/848] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:18.006714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51241805-0217-421b-b089-c3162083f436 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.007003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51241805-0217-421b-b089-c3162083f436 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.007285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51241805-0217-421b-b089-c3162083f436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.023553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7e6cc-7c55-43a0-9aff-4bab8df09ddb None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:18.027132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-51241805-0217-421b-b089-c3162083f436 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:18.028129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-51241805-0217-421b-b089-c3162083f436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:18.070609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:18.071720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.072220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.072706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.073545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:18.074057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:18.075625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:18.089764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 429/849] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:18.095644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:18.124574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 421/850] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:18.212554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146a0be-034d-4a44-983c-2478c57c63dd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:18.227835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:18.249669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:18.250309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.250619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.250893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.258263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:18.259722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:18.272611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:18.273544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:18.274424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 430/851] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:18.279815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51241805-0217-421b-b089-c3162083f436 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:18.317818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:18.397738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51241805-0217-421b-b089-c3162083f436 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:18.407837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:18.532720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219abade-2461-4786-8e0a-104ba4bb4391 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:18.564734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:18.565453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.565797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.566132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.566891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:18.567261 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:18.568797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:18.580563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c73d2-b63f-400f-9e7e-81e8fc164c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:18.581202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c73d2-b63f-400f-9e7e-81e8fc164c8f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.581481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c73d2-b63f-400f-9e7e-81e8fc164c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.581754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c73d2-b63f-400f-9e7e-81e8fc164c8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.584271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 422/852] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:18.593618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:18.596859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c73d2-b63f-400f-9e7e-81e8fc164c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-882c73d2-b63f-400f-9e7e-81e8fc164c8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:18.603698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:18.604863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:19:18.605749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:18.608688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 431/853] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:19:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:18.615573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c73d2-b63f-400f-9e7e-81e8fc164c8f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:18.653665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:18.709149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:18.715152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b186993-8739-4429-a6d5-054a94295c20 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:18.739801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c73d2-b63f-400f-9e7e-81e8fc164c8f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:18.757651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9d2ea-8c5a-4bce-925f-305a2698e588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:18.758256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d2ea-8c5a-4bce-925f-305a2698e588 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.758537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d2ea-8c5a-4bce-925f-305a2698e588 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.758812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d2ea-8c5a-4bce-925f-305a2698e588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.759549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9d2ea-8c5a-4bce-925f-305a2698e588 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=bd886328304543828dca35df3a0b7411) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:18.759865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9d2ea-8c5a-4bce-925f-305a2698e588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:18.761320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9d2ea-8c5a-4bce-925f-305a2698e588 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:18.768806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 423/854] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:18 2022] GET /identity/v3/role_assignments?user.id=bd886328304543828dca35df3a0b7411&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:18.770256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45b089-c83b-4739-bd5e-5b7309c35732 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.770620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45b089-c83b-4739-bd5e-5b7309c35732 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.770967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45b089-c83b-4739-bd5e-5b7309c35732 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.777604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 424/855] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:18.780124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d231a85-baba-49c0-932a-3da840eb14dd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.780501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d231a85-baba-49c0-932a-3da840eb14dd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.780775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d231a85-baba-49c0-932a-3da840eb14dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.799582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:18.800652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.801138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.801755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:18.802668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:18.803196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:18.804681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2d231a85-baba-49c0-932a-3da840eb14dd None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:18.805326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:18.805917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2d231a85-baba-49c0-932a-3da840eb14dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:18.816947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 432/856] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:19:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:18.823609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:18.912209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 425/857] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:18.953090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:18.959486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4e519-2271-44f1-9171-03d3667d84e5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:18.997897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:18.998572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:18.998855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:18.999133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.011887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:19.013461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:19.030149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:19.031138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:19.032057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:19.032057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:19.032057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:19.032057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 433/858] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:19.037735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d231a85-baba-49c0-932a-3da840eb14dd None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:19.139184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:19.150220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d231a85-baba-49c0-932a-3da840eb14dd None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:19.193867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:19.194797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.195264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.195721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.196615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:19.197125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:19.198911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:19.212663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 426/859] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:19.217163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:19.218883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:19.340207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854911e-caf7-44fe-8066-ace82e6a1026 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:19.357796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbb79d-e005-490a-ab01-717dc1b75218 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:19.391998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:19.393020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.393526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.394011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.404193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:19.404864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.405147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.405424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.405915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:19.406022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:19.407510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:19.407927 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:19.421523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:19.422554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:19.423796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:19.424946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:19.426049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:19.426920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:19.428202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 427/860] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:19:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:19.429059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:19.430157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 434/861] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 13:19:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:19.437504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:19.447700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:19.455995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:19.555555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95030-a6cb-45fd-8849-49f155aeb3c7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:19.581373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b16d3-ac1c-4ebb-80ef-f58458f85549 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:19.599414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:19.600053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.600479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.600816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.601605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:19.602014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:19.603827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:19.623167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:19.623888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.624259 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.624597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.625326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:19.626242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 428/862] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 13:19:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:19.628456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:19.630068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:19.632064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:19.640263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 435/863] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:19.642856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:19.715849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:19.737261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:19.756029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae83bb0-b5fb-4a39-bed3-4f9fb658240d None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:19.770423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5708a-45bd-4204-b7eb-af3a55aca459 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:19.799800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437eda6-c431-492c-a86a-5dbba4cc3927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:19.800453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437eda6-c431-492c-a86a-5dbba4cc3927 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.800743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437eda6-c431-492c-a86a-5dbba4cc3927 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.801017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437eda6-c431-492c-a86a-5dbba4cc3927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.818844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b40ce-38d8-491d-a82c-c21bd942128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:19.819856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b40ce-38d8-491d-a82c-c21bd942128c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.820246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b40ce-38d8-491d-a82c-c21bd942128c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.820525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b40ce-38d8-491d-a82c-c21bd942128c None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.821164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b40ce-38d8-491d-a82c-c21bd942128c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:19.821463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b40ce-38d8-491d-a82c-c21bd942128c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:19.822836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b40ce-38d8-491d-a82c-c21bd942128c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:19.832083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 436/864] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:19.833311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035e84e-d6c3-46e8-a8e5-30be83e06747 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.833605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035e84e-d6c3-46e8-a8e5-30be83e06747 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.833878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035e84e-d6c3-46e8-a8e5-30be83e06747 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.835290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 437/865] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:19.842770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a962c95-0ec7-4b7c-b4f0-984c6379275c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:19.843033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a962c95-0ec7-4b7c-b4f0-984c6379275c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:19.843319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a962c95-0ec7-4b7c-b4f0-984c6379275c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:19.878925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8a962c95-0ec7-4b7c-b4f0-984c6379275c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:19.880626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8a962c95-0ec7-4b7c-b4f0-984c6379275c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:19.915648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437eda6-c431-492c-a86a-5dbba4cc3927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:19.917139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437eda6-c431-492c-a86a-5dbba4cc3927 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:19.982798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 438/866] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:20.019118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:20.062067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 429/867] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 13:19:20.068878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a962c95-0ec7-4b7c-b4f0-984c6379275c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:20.141947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:20.178838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a962c95-0ec7-4b7c-b4f0-984c6379275c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:20.218705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:20.220431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:20.221293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:20.222064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:20.237773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:20.239921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:20.260962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:20.262057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:20.263027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:20.263027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:20.263027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:20.265260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 439/868] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:20.276202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437eda6-c431-492c-a86a-5dbba4cc3927 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:20.294518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:20.392206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437eda6-c431-492c-a86a-5dbba4cc3927 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:20.429960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:20.430496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:20.430676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cd5db-0c6c-4d4a-8bc9-c26c8c05500b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:20.430859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:20.431088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:20.431746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:20.432139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:20.433673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:20.451393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 430/869] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:20.452838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:20.454497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:20.470181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233c8a5-8c30-4f52-b2af-953cd8c75950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:20.470890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c8a5-8c30-4f52-b2af-953cd8c75950 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:20.471294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c8a5-8c30-4f52-b2af-953cd8c75950 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:20.471603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233c8a5-8c30-4f52-b2af-953cd8c75950 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:20.472328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233c8a5-8c30-4f52-b2af-953cd8c75950 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:20.473784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233c8a5-8c30-4f52-b2af-953cd8c75950 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:20.489369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:20.572369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 440/870] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:19:20 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:19:20.580087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fe548-403f-4cc1-887c-d0cc34d936fd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:20.631174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:20.632704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:20.633206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:20.633689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:20.643156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:20.645205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:20.660283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:20.661267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:20.662164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:20.662164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:20.662164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:20.662164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:20.662164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:20.662164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:20.662164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:20.662164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:20.664802 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 431/871] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:20.673940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233c8a5-8c30-4f52-b2af-953cd8c75950 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:20.803836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233c8a5-8c30-4f52-b2af-953cd8c75950 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:20.842752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410cc7-18df-49a5-bff7-e8e00335108f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:20.843745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410cc7-18df-49a5-bff7-e8e00335108f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:20.845004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410cc7-18df-49a5-bff7-e8e00335108f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:20.845608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410cc7-18df-49a5-bff7-e8e00335108f None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:20.846593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410cc7-18df-49a5-bff7-e8e00335108f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:20.847110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410cc7-18df-49a5-bff7-e8e00335108f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:20.848798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410cc7-18df-49a5-bff7-e8e00335108f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:20.867543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 441/872] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:20.875926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:20.923488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa19150-a896-4354-b984-d5e91184c2da None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:20.923811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa19150-a896-4354-b984-d5e91184c2da None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:20.924072 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa19150-a896-4354-b984-d5e91184c2da None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:20.925730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 442/873] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:20.931124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b6e81-1c0b-47f5-b453-a816fc98390b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:20.931400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b6e81-1c0b-47f5-b453-a816fc98390b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:20.931680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b6e81-1c0b-47f5-b453-a816fc98390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:20.960803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-410b6e81-1c0b-47f5-b453-a816fc98390b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:20.962465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-410b6e81-1c0b-47f5-b453-a816fc98390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:21.016032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:21.027766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:21.051669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7640-e140-4fd6-8437-f0eb73a37950 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:21.072703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 443/874] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:21.093246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:21.094722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:21.095327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:21.095917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:21.096863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=b56fb09428cb4ab2b49ca199058a5a7a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:21.097345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:21.098939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:21.106247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 432/875] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:20 2022] GET /identity/v3/role_assignments?user.id=b56fb09428cb4ab2b49ca199058a5a7a&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:21.345954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b6e81-1c0b-47f5-b453-a816fc98390b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:21.469128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b6e81-1c0b-47f5-b453-a816fc98390b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:21.514449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:21.515098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:21.515378 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:21.515650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:21.516810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:21.518399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:21.527952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:21.529066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:21.529904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 444/876] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:21.537323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:21.662485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb3e00-d86f-415b-ad5f-f7ab6c9cb2d3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:21.701325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb93be8-9b61-42ce-99f9-2a2be7b5a072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:21.702341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb93be8-9b61-42ce-99f9-2a2be7b5a072 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:21.702858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb93be8-9b61-42ce-99f9-2a2be7b5a072 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:21.703339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb93be8-9b61-42ce-99f9-2a2be7b5a072 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:21.704254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb93be8-9b61-42ce-99f9-2a2be7b5a072 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:21.704864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb93be8-9b61-42ce-99f9-2a2be7b5a072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:21.706475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb93be8-9b61-42ce-99f9-2a2be7b5a072 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:21.712010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 433/877] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:21.717333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:21.763388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:21.846402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b36c-406c-41db-9c94-e758ca4517bc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:21.887303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0660bc-e579-470b-abe6-10969e3bd2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:21.888639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0660bc-e579-470b-abe6-10969e3bd2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:21.889320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0660bc-e579-470b-abe6-10969e3bd2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:21.889903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0660bc-e579-470b-abe6-10969e3bd2c3 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:21.891141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0660bc-e579-470b-abe6-10969e3bd2c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:21.891790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0660bc-e579-470b-abe6-10969e3bd2c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:21.894035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0660bc-e579-470b-abe6-10969e3bd2c3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:21.900837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 445/878] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:22.007183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29704159-27c8-493e-9ebd-7ade2c3cfd1b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:22.007363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29704159-27c8-493e-9ebd-7ade2c3cfd1b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:22.007800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29704159-27c8-493e-9ebd-7ade2c3cfd1b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:22.009453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 434/879] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:22.015397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b68a9-f5f7-4d6c-935c-b7938ef9cc0e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:22.016475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b68a9-f5f7-4d6c-935c-b7938ef9cc0e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:22.019789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b68a9-f5f7-4d6c-935c-b7938ef9cc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:22.043495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2b2b68a9-f5f7-4d6c-935c-b7938ef9cc0e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:22.044800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2b2b68a9-f5f7-4d6c-935c-b7938ef9cc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:22.127029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 446/880] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:22 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:22.413403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29704159-27c8-493e-9ebd-7ade2c3cfd1b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:22.429616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:22.537499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29704159-27c8-493e-9ebd-7ade2c3cfd1b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:22.575013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77481138-7142-49db-ba45-849d14717cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:22.575617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77481138-7142-49db-ba45-849d14717cc9 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:22.575908 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77481138-7142-49db-ba45-849d14717cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:22.576199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77481138-7142-49db-ba45-849d14717cc9 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:22.576862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77481138-7142-49db-ba45-849d14717cc9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:22.578228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77481138-7142-49db-ba45-849d14717cc9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:22.650865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 435/881] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:19:22 2022] POST /identity/v3/services => generated 269 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:22.816011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98dddb-fa6a-4591-b56e-223477f30be4 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:22.817091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98dddb-fa6a-4591-b56e-223477f30be4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:22.817582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98dddb-fa6a-4591-b56e-223477f30be4 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:22.819625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 447/882] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:22.825069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4a72f-82de-4f19-9d0a-c1e192f969a2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:22.825973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4a72f-82de-4f19-9d0a-c1e192f969a2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:22.826456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4a72f-82de-4f19-9d0a-c1e192f969a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:22.848640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d4e4a72f-82de-4f19-9d0a-c1e192f969a2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:22.849815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d4e4a72f-82de-4f19-9d0a-c1e192f969a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:22.928717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:22.936324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 436/883] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:23.217573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98dddb-fa6a-4591-b56e-223477f30be4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:23.250832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:23.290361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3656f-c7ef-4cb0-9270-ed2b71d512af None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:23.290909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3656f-c7ef-4cb0-9270-ed2b71d512af None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:23.291407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3656f-c7ef-4cb0-9270-ed2b71d512af None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:23.294380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 437/884] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:23.299887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca810f-ead9-47af-88cd-3f3219979fa3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:23.300508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca810f-ead9-47af-88cd-3f3219979fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:23.301005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca810f-ead9-47af-88cd-3f3219979fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:23.323077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0cca810f-ead9-47af-88cd-3f3219979fa3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:23.324654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0cca810f-ead9-47af-88cd-3f3219979fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:23.340688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98dddb-fa6a-4591-b56e-223477f30be4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:23.392098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:23.393428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:23.394032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:23.394629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:23.396113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:23.399065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:23.411157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:23.412221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:23.413063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 448/885] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:23.421522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:23.440324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 438/886] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:23.532578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:23.537887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbdf8e-2497-4a05-a255-a9732f5bc319 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:23.573300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:23.574630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:23.574908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:23.575178 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:23.575811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:23.576111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:23.577499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:23.583375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 449/887] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:23.589560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca810f-ead9-47af-88cd-3f3219979fa3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:23.702924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca810f-ead9-47af-88cd-3f3219979fa3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:23.715139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:23.737849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:23.746654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:23.748391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:23.748391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:23.748391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:23.748642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:23.748891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:23.750981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:23.756987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 439/888] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:23.763292 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:23.862714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5b333-130b-4196-b263-c793809203bf None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:23.894875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e513ae-5d0d-40fa-93f7-30d4ab60d072 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:23.915548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856d3b8-3f36-44e5-8010-d463fceda5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:23.916583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856d3b8-3f36-44e5-8010-d463fceda5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:23.917073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856d3b8-3f36-44e5-8010-d463fceda5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:23.917568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856d3b8-3f36-44e5-8010-d463fceda5c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:23.933802 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:23.934379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856d3b8-3f36-44e5-8010-d463fceda5c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:23.934511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:23.934675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:23.934996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:23.935817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] RBAC: Adding query filter params (interface=public, service_id=5dc55b3c30dd447baf4f1e4284de27a4, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:23.936124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:23.937568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b856d3b8-3f36-44e5-8010-d463fceda5c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:23.942607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:23.943582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:23.944486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:23.952161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 440/890] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 13:19:23 2022] GET /identity/v3/endpoints?service_id=5dc55b3c30dd447baf4f1e4284de27a4&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:23.952440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 450/890] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 13:19:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:23.953555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:23.954540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856d3b8-3f36-44e5-8010-d463fceda5c6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:23.980598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:24.022442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:24.069693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e26fe8-f778-4689-8586-8f9ba6e0a3e8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:24.077379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856d3b8-3f36-44e5-8010-d463fceda5c6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:24.108697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d3ce5-b761-409d-806d-a8c8dbcb22f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:24.109316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d3ce5-b761-409d-806d-a8c8dbcb22f0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.110218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d3ce5-b761-409d-806d-a8c8dbcb22f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.111100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d3ce5-b761-409d-806d-a8c8dbcb22f0 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.111720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d3ce5-b761-409d-806d-a8c8dbcb22f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:24.113176 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d3ce5-b761-409d-806d-a8c8dbcb22f0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:24.117695 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:24.118493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.118607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.118896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.124435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 441/891] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:19:23 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:19:24.124620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:24.124724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102c90d-037b-4831-884f-687362aae79d None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.124724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102c90d-037b-4831-884f-687362aae79d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.124876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:24.124946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102c90d-037b-4831-884f-687362aae79d None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.125020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:24.127826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 442/892] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:24.129319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8bc8-e834-4ddb-b805-42ad5e3c4566 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.129614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8bc8-e834-4ddb-b805-42ad5e3c4566 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.129895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc8bc8-e834-4ddb-b805-42ad5e3c4566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.137460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 451/893] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 13:19:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:24.144273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:24.154337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-aacc8bc8-e834-4ddb-b805-42ad5e3c4566 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:24.155574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-aacc8bc8-e834-4ddb-b805-42ad5e3c4566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:24.252713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 443/894] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:24.254268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:24.278292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cd7e7-8a86-46be-95c0-ec16185428ab None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:24.319933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:24.320619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.320962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.321245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.334157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:24.335745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:24.353314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:24.354363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:24.355248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:24.355248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:24.355248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:24.355248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 452/895] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:24.362403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc8bc8-e834-4ddb-b805-42ad5e3c4566 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:24.422722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:24.478894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc8bc8-e834-4ddb-b805-42ad5e3c4566 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:24.518186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:24.518800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.519084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.519360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.520009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:24.520348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:24.521816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:24.533369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:24.535439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 444/896] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:24.544515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:24.648214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda7268-b30f-4bf7-9f6e-e944677df70c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:24.656233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c435d-15fa-4eb1-b7bb-3426f4e46594 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:24.668617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:24.685308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:24.693296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7876ec4-f797-4462-a436-3747ee432b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:24.694385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7876ec4-f797-4462-a436-3747ee432b9a None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.697279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7876ec4-f797-4462-a436-3747ee432b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.697279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7876ec4-f797-4462-a436-3747ee432b9a None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.697279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7876ec4-f797-4462-a436-3747ee432b9a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:24.697530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:24.697691 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.697967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.698245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.698696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7876ec4-f797-4462-a436-3747ee432b9a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:24.708384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:24.709975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:24.722753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:24.723811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:24.724867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:24.725937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 445/897] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:19:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:24.729732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:24.776713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 453/898] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:19:24 2022] POST /identity/v3/services => generated 241 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:24.777984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de91ba-823b-42b4-bff2-14d21974700a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.778312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de91ba-823b-42b4-bff2-14d21974700a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.778552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de91ba-823b-42b4-bff2-14d21974700a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.780092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 454/899] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:24.789649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec8f2-b65a-4d90-b53c-e5c0ed7aff06 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.789993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec8f2-b65a-4d90-b53c-e5c0ed7aff06 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.790227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec8f2-b65a-4d90-b53c-e5c0ed7aff06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.817730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e7bec8f2-b65a-4d90-b53c-e5c0ed7aff06 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:24.819532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e7bec8f2-b65a-4d90-b53c-e5c0ed7aff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:24.868787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0dee-838c-4a01-a1b6-05eff4e5a7a5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:24.911720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:24.912846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 455/900] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:24.917021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:24.917603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:24.918078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:24.919200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:24.919755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:24.921733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:24.946218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 446/901] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 13:19:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:24.952528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec8f2-b65a-4d90-b53c-e5c0ed7aff06 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:25.067629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec8f2-b65a-4d90-b53c-e5c0ed7aff06 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:25.086859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:25.113352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c619fcd-d4fb-4d57-a145-48daa2dcea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:25.113974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c619fcd-d4fb-4d57-a145-48daa2dcea57 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:25.114257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c619fcd-d4fb-4d57-a145-48daa2dcea57 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:25.114532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c619fcd-d4fb-4d57-a145-48daa2dcea57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:25.191621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c619fcd-d4fb-4d57-a145-48daa2dcea57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:25.193140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c619fcd-d4fb-4d57-a145-48daa2dcea57 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:25.213509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:25.259203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:25.334345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 456/902] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:24 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:19:25.340208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c619fcd-d4fb-4d57-a145-48daa2dcea57 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:25.360738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:25.365170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d98d0-0b36-4cb9-9d15-97b161c58bb7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:25.405883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:25.407226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:25.407634 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:25.407985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:25.409173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:25.410212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:25.419755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:25.420982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:25.422095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:25.428506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 447/903] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 13:19:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:25.429904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:25.468322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c619fcd-d4fb-4d57-a145-48daa2dcea57 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:25.510329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:25.511111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:25.511508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:25.511877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:25.524632 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:25.526985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:25.534688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:25.544044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b693a3-9c91-4ae9-a08a-7c3825c6acdf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:25.545815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:25.550555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:25.551447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:25.551447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:25.551447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:25.551447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 457/904] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:25.553789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:25.588699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:25.589330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:25.589612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:25.589885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:25.590800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:25.591103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:25.593167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:25.605894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 448/905] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:25.609246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:25.644604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:25.691363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b03be-d243-4650-80d6-3b47dcd88687 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:25.725809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eb7e3-3073-48ea-ad97-e793798e3a88 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:25.733885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:25.735018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:25.735306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:25.735571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:25.736241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:25.736547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:25.738635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:25.760350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 458/906] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:25.763141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:25.780419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:25.781031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:25.781339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:25.781627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:25.782534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:25.782712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:25.784039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:25.799236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 449/907] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:25.801893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:25.854668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:25.889127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715a080-de26-4909-8ae8-b1694a498dc6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:25.920890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:25.930812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:25.931885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:25.932406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:25.932928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:25.942712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:25.945448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:25.957655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac33473-432c-4970-9e6e-08a4969691fe None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:25.962874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:25.963721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:25.964980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:25.964980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:25.964980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:25.964980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:25.964980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:25.964980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:25.964980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:25.964980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:25.973170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 459/908] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:25.976327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:26.027121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:26.028241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.028752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.029319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.030631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] RBAC: Adding query filter params (interface=public, service_id=ba153e4d9ec54dd18db558394767b079, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:26.031207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:26.034612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:26.045217 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 450/909] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 13:19:25 2022] GET /identity/v3/endpoints?service_id=ba153e4d9ec54dd18db558394767b079&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:26.051110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:26.107431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03f58a-5f6e-47d7-bd17-f80959419b1a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:26.149982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:26.151365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.152019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.152596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.153481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:26.153961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:26.155558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:26.178957 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 460/910] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:26.180243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:26.191130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:26.211610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8ec5c-bff1-4590-8872-d7ec88408bfa None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:26.254156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6c7d5-7824-4875-bda7-64c914c7abf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:26.254784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6c7d5-7824-4875-bda7-64c914c7abf4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.255146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6c7d5-7824-4875-bda7-64c914c7abf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.255365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:26.255608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6c7d5-7824-4875-bda7-64c914c7abf4 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.256357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6c7d5-7824-4875-bda7-64c914c7abf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:26.257887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6c7d5-7824-4875-bda7-64c914c7abf4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:26.271317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 451/911] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:19:26 2022] GET /identity/v3/services => generated 1497 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:19:26.299054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97162f-aaab-4415-a243-9864eff2e494 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.299435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97162f-aaab-4415-a243-9864eff2e494 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.299626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97162f-aaab-4415-a243-9864eff2e494 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.301334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 452/912] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:26.309110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c86fc-9380-494d-bb2a-ea3cba1f1e1d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.309449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c86fc-9380-494d-bb2a-ea3cba1f1e1d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.310019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c86fc-9380-494d-bb2a-ea3cba1f1e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.323974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe3e4e-acdb-440b-95a3-cd11a8b908e7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:26.332042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-788c86fc-9380-494d-bb2a-ea3cba1f1e1d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:26.333363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-788c86fc-9380-494d-bb2a-ea3cba1f1e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:26.371287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:26.372342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.372835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.373317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.374318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] RBAC: Adding query filter params (role.id=2a934a9192c84ef3a65ed7acfc442b2f, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=b56fb09428cb4ab2b49ca199058a5a7a) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:26.374912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:26.377544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:26.384084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 461/913] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:26 2022] GET /identity/v3/role_assignments?user.id=b56fb09428cb4ab2b49ca199058a5a7a&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=2a934a9192c84ef3a65ed7acfc442b2f => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:26.455780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 453/914] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:26.740455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:26.859550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31fbbb-3592-47ae-9631-7915507fcb7e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:26.875709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:26.897918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48af9ce-6b99-428e-8367-16a97bec66fe None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.898329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48af9ce-6b99-428e-8367-16a97bec66fe None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.898460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48af9ce-6b99-428e-8367-16a97bec66fe None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.899800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 454/915] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:26.902222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:26.902869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.903148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.903418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.904447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:26.905436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef415e-b105-41bf-9b49-abec22b508a8 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:26.905719 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef415e-b105-41bf-9b49-abec22b508a8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:26.905893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:26.906001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef415e-b105-41bf-9b49-abec22b508a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:26.916909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:26.918018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:26.919323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 462/916] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:26.925337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:26.927895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-56ef415e-b105-41bf-9b49-abec22b508a8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:26.929247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-56ef415e-b105-41bf-9b49-abec22b508a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:27.031125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 455/917] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:27.050478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06129526-f658-45b7-b798-9defccdbcd76 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:27.092166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:27.092931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:27.093210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:27.093657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:27.095263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:27.095578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:27.096999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:27.102921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 463/918] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:27.108848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef415e-b105-41bf-9b49-abec22b508a8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:27.120790 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:27.224157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef415e-b105-41bf-9b49-abec22b508a8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:27.267118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:27.267788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:27.268075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:27.268377 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:27.269029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:27.269396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:27.270793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:27.277134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 456/919] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 13:19:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:27.282860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:27.334461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:27.359167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:27.418762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c6f81-db7c-45f8-8358-704fec288de5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:27.458489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97f455-4c88-48c9-aca0-9c4d043e7cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:27.459585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97f455-4c88-48c9-aca0-9c4d043e7cac None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:27.460098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97f455-4c88-48c9-aca0-9c4d043e7cac None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:27.460620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97f455-4c88-48c9-aca0-9c4d043e7cac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:27.461518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97f455-4c88-48c9-aca0-9c4d043e7cac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:27.463218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b97f455-4c88-48c9-aca0-9c4d043e7cac None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:27.476565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:27.491700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb283d6-2202-4f44-91f4-d70ed2538543 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:27.532150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:27.532838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:27.533730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:27.533948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:27.534540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:27.536068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:27.545419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:27.549244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:27.550115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 457/920] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:27.553657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:27.628574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 464/921] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:19:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:27.672057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735bf8-4c1a-47b4-999b-16a035b259f5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:27.716886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:27.723319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:27.724034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:27.724398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:27.724674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:27.725309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:27.725617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:27.727008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:27.736284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 458/922] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:27.739715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97f455-4c88-48c9-aca0-9c4d043e7cac None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:27.852588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:27.859340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97f455-4c88-48c9-aca0-9c4d043e7cac None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:27.901078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:27.901700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:27.902372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:27.902889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:27.903866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:27.904450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:27.906667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:27.912967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 465/923] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:27.918995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:28.035970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0d7b-0fa0-4fa8-b256-6397f47a00db None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:28.074307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772f159-93dc-4fa3-a800-099287915c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:28.074915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772f159-93dc-4fa3-a800-099287915c3d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:28.075199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772f159-93dc-4fa3-a800-099287915c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:28.075472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772f159-93dc-4fa3-a800-099287915c3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:28.076196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772f159-93dc-4fa3-a800-099287915c3d None admin] RBAC: Adding query filter params (interface=public, service_id=aa1d51a11d234802b31c28437dfebe01, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:28.076514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772f159-93dc-4fa3-a800-099287915c3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:28.077879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772f159-93dc-4fa3-a800-099287915c3d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:28.085221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 459/924] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 13:19:27 2022] GET /identity/v3/endpoints?service_id=aa1d51a11d234802b31c28437dfebe01&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:28.090578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:28.207612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d41cf-c68e-4aec-a8c6-98a572a7fa10 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:28.247926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6b291-166d-4ad6-8f37-b2627f6a8cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:28.248567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6b291-166d-4ad6-8f37-b2627f6a8cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:28.248852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6b291-166d-4ad6-8f37-b2627f6a8cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:28.249129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6b291-166d-4ad6-8f37-b2627f6a8cd8 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:28.249749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6b291-166d-4ad6-8f37-b2627f6a8cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:28.251222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6b291-166d-4ad6-8f37-b2627f6a8cd8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:28.264349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 466/925] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:19:28 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:19:28.381988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fdaaf-be78-4a30-b23f-7e500a9b9120 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:28.382330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fdaaf-be78-4a30-b23f-7e500a9b9120 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:28.382516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fdaaf-be78-4a30-b23f-7e500a9b9120 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:28.387062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 460/926] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:28.390045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96a21-0462-471d-b4dc-76cc62b4f059 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:28.390325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96a21-0462-471d-b4dc-76cc62b4f059 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:28.390625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c96a21-0462-471d-b4dc-76cc62b4f059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:28.411476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f2c96a21-0462-471d-b4dc-76cc62b4f059 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:28.412732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2c96a21-0462-471d-b4dc-76cc62b4f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:28.437434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:28.458134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc44ad1-23e1-4967-8fff-e61a923491d9 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:28.458347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc44ad1-23e1-4967-8fff-e61a923491d9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:28.458621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc44ad1-23e1-4967-8fff-e61a923491d9 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:28.460278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 461/927] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:28.465405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961d7b7-e0b0-4b3e-bf41-55da6a3c8180 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:28.465692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961d7b7-e0b0-4b3e-bf41-55da6a3c8180 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:28.465971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961d7b7-e0b0-4b3e-bf41-55da6a3c8180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:28.488366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0961d7b7-e0b0-4b3e-bf41-55da6a3c8180 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:28.489579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0961d7b7-e0b0-4b3e-bf41-55da6a3c8180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:28.498536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 467/928] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:28.553420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:28.565829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 462/929] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:28.780281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c96a21-0462-471d-b4dc-76cc62b4f059 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:28.853885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961d7b7-e0b0-4b3e-bf41-55da6a3c8180 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:28.899538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c96a21-0462-471d-b4dc-76cc62b4f059 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:28.948938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:28.949580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:28.949862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:28.950135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:28.950732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:28.952305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:28.962610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:28.963576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:28.964702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 468/930] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 13:19:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:28.973640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:28.977163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961d7b7-e0b0-4b3e-bf41-55da6a3c8180 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:28.997484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:29.027816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547128e5-841c-433f-a2fb-5322b7e99f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:29.028514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547128e5-841c-433f-a2fb-5322b7e99f82 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.028799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547128e5-841c-433f-a2fb-5322b7e99f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.029076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547128e5-841c-433f-a2fb-5322b7e99f82 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.044702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547128e5-841c-433f-a2fb-5322b7e99f82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-547128e5-841c-433f-a2fb-5322b7e99f82 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:29.052520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:29.053518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:29.054340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:29.059409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 463/931] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 13:19:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:29.060537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547128e5-841c-433f-a2fb-5322b7e99f82 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:29.100948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1839c-f88a-480f-a801-5061c6481fa5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:29.131306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:29.139185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:29.140103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.140382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.140911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.141726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:29.142239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:29.143764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:29.152940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 469/932] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:29.155376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:29.197830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547128e5-841c-433f-a2fb-5322b7e99f82 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:29.247363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:29.248371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.248805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.249669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.251903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:29.252431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:29.254572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:29.260636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:29.272417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 464/933] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 13:19:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:29.278042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:29.289186 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fcb10-802b-4ff8-9a61-efeef7b95970 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:29.347619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:29.348817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.349270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.349847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.350649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:29.351392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:29.352952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:29.360941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 470/934] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:29.367561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:29.428220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4373697-58c0-49c7-86e0-86c31eb05fab None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:29.447096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:29.471178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:29.471823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.472106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.472411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.484872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:29.486478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:29.503508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:29.504525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:29.505527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:29.505527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:29.505527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:29.509509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 465/935] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:29.510454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:29.523905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a430f-5409-4805-8af8-29782ea11a87 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:29.576431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c202-e2f7-44a6-b4ec-46ede1e73bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:29.577618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c202-e2f7-44a6-b4ec-46ede1e73bfc None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.579175 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c202-e2f7-44a6-b4ec-46ede1e73bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.579975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c202-e2f7-44a6-b4ec-46ede1e73bfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.581128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560c202-e2f7-44a6-b4ec-46ede1e73bfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:29.583416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560c202-e2f7-44a6-b4ec-46ede1e73bfc None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:29.634957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a54f38-1b8e-416d-97a1-e07a2cb1404a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:29.674057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91709bab-4f49-48db-9d0f-898be6717232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:29.674742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91709bab-4f49-48db-9d0f-898be6717232 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.675024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91709bab-4f49-48db-9d0f-898be6717232 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.675300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91709bab-4f49-48db-9d0f-898be6717232 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.676030 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91709bab-4f49-48db-9d0f-898be6717232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:29.676425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91709bab-4f49-48db-9d0f-898be6717232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:29.677985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91709bab-4f49-48db-9d0f-898be6717232 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:29.689010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 471/936] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:19:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:29.695704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 466/937] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:29.702030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c202-e2f7-44a6-b4ec-46ede1e73bfc None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:29.708610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:29.712889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e14b42-25d3-401a-ad58-3e3e51a6a633 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.713239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e14b42-25d3-401a-ad58-3e3e51a6a633 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.713521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e14b42-25d3-401a-ad58-3e3e51a6a633 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.716203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 467/938] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:29.724851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca4a33-3345-4cd1-833e-3dd1e0d39dfd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.725457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca4a33-3345-4cd1-833e-3dd1e0d39dfd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.725971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca4a33-3345-4cd1-833e-3dd1e0d39dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.761224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-63ca4a33-3345-4cd1-833e-3dd1e0d39dfd None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:29.762969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-63ca4a33-3345-4cd1-833e-3dd1e0d39dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:29.793215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:29.889267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c202-e2f7-44a6-b4ec-46ede1e73bfc None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:29.900716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 468/939] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:29.940896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:29.941848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:29.942314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:29.942689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:29.955836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:29.956036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:29.974281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:29.978638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:29.979590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 472/940] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:19:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:29.983253 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca4a33-3345-4cd1-833e-3dd1e0d39dfd None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:30.113307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca4a33-3345-4cd1-833e-3dd1e0d39dfd None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:30.162665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:30.163597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:30.164080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:30.164606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:30.165986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:30.166509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:30.169029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:30.198286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 469/941] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 13:19:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:30.205392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:30.245296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:30.270520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:30.314111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:30.379506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149d51-9ed3-4976-9ee8-b0071b89307f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:30.396956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce27da2-cef1-4388-813b-ff5f8bf6bd3b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:30.425106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:30.425653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:30.425992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:30.426262 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:30.441002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:30.441267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:30.441911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:30.442198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:30.442472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:30.442658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:30.443138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:30.444637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:30.447301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING keystone.server.flask.common [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:30.461266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:30.462310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:30.463218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:30.463218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:30.463218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:30.463218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 473/942] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:30.469652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:30.509868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.sql.core [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 13:19:30.509868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 13:19:30.509868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [parameters: {'id': '9ab7180b438c4889844d8fb51f2697d5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 13:19:30.509868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81685) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3c53f888-44f8-4ccf-bbf0-63c19273c570 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9ab7180b438c4889844d8fb51f2697d5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9ab7180b438c4889844d8fb51f2697d5. Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application conn.query(q) Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 13:19:30.529617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result.read() Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 13:19:30.530817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application next(self.gen) Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application yield resource Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application next(self.gen) Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise self.value Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application session.commit() Aug 24 13:19:30.533315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.session.flush() Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application compat.raise_( Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 13:19:30.534228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 13:19:30.535254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application conn.query(q) Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application result.read() Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 13:19:30.536048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [parameters: {'id': '9ab7180b438c4889844d8fb51f2697d5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:30.539186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:30.540757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9ab7180b438c4889844d8fb51f2697d5. Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 470/943] 10.222.0.73 () {60 vars in 1140 bytes} [Wed Aug 24 13:19:30 2022] POST /identity/v3/projects => generated 262 bytes in 286 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213fc87-e56c-4611-be93-a7ad5849e32a None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:30.543075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213fc87-e56c-4611-be93-a7ad5849e32a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:30.544475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213fc87-e56c-4611-be93-a7ad5849e32a None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:30.544475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 471/944] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:30.545538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213fc87-e56c-4611-be93-a7ad5849e32a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:30.632952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd58617-c135-44ba-96b4-1a448e334eee None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:30.646411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:30.653523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:30.683457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b21dc1-1d7e-4a85-8e29-b581cd771d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:30.684136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b21dc1-1d7e-4a85-8e29-b581cd771d45 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:30.685439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b21dc1-1d7e-4a85-8e29-b581cd771d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:30.685439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b21dc1-1d7e-4a85-8e29-b581cd771d45 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:30.685439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b21dc1-1d7e-4a85-8e29-b581cd771d45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:30.690844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b21dc1-1d7e-4a85-8e29-b581cd771d45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:30.690844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b21dc1-1d7e-4a85-8e29-b581cd771d45 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:30.699613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213fc87-e56c-4611-be93-a7ad5849e32a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:30.704281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 474/945] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:30.705504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bada-0da1-4481-9692-9c7a10efb779 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:30.705884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bada-0da1-4481-9692-9c7a10efb779 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:30.706231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bada-0da1-4481-9692-9c7a10efb779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:30.728791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2d64bada-0da1-4481-9692-9c7a10efb779 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:30.730533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2d64bada-0da1-4481-9692-9c7a10efb779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:30.764497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:30.765161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:30.765447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:30.765777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:30.776270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:30.778200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:30.804384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:30.805852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:30.806959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:30.806959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:30.806959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:30.806959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:30.806959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:30.806959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:30.806959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:30.806959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:30.808512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 472/946] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:30.809578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:30.842242 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 475/947] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:30.849290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64bada-0da1-4481-9692-9c7a10efb779 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:30.966034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f06cc5-5b2a-48ec-a3fb-49a12110967a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:31.006034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:31.007095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:31.007576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:31.008079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:31.009827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:31.013930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64bada-0da1-4481-9692-9c7a10efb779 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:31.020421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:31.023210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:31.025627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:31.043044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:31.044314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:31.045435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:31.045435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:31.045435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:31.045435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:31.045435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:31.045435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:31.045435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:31.045435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:31.049241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 473/948] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:31.055627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:31.063306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:31.063932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:31.064235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:31.064512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:31.065267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:31.065578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:31.067033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:31.096237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 476/949] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:31.138061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:31.217211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f7d6c-2866-4f68-a908-15ea58d68bd4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:31.268904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:31.270225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:31.270710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:31.271090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:31.272340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:31.272898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:31.275329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:31.287965 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a2fbf-ce87-4f73-a363-8af5e82a09f6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:31.292260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 474/950] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:31.299428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:31.342650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:31.343316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:31.345076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:31.345076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:31.345076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:31.346684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:31.362147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:31.363608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:31.364563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 477/951] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:31.369425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:31.371351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:31.407798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:31.444152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11832-068b-46d7-935f-80cd259444ff None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:31.484501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500bbd5-81f2-4947-b1da-06afeb4b35c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:31.485618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500bbd5-81f2-4947-b1da-06afeb4b35c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:31.486125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500bbd5-81f2-4947-b1da-06afeb4b35c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:31.486579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500bbd5-81f2-4947-b1da-06afeb4b35c5 None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/b56fb09428cb4ab2b49ca199058a5a7a/roles/2a934a9192c84ef3a65ed7acfc442b2f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:31.489255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500bbd5-81f2-4947-b1da-06afeb4b35c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=b56fb09428cb4ab2b49ca199058a5a7a, role_id=2a934a9192c84ef3a65ed7acfc442b2f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:31.490885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500bbd5-81f2-4947-b1da-06afeb4b35c5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:31.502861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd8e92-2cc6-47bd-82eb-a4bee8a17530 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:31.541741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:31.542356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:31.542639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:31.542912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:31.543558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:31.543864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:31.545320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:31.552374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 478/952] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:31.558087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:31.600422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 475/953] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:19:31 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/b56fb09428cb4ab2b49ca199058a5a7a/roles/2a934a9192c84ef3a65ed7acfc442b2f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:19:31.660913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:31.676244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ca3e6-367e-4c63-ab04-dde236814c00 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:31.718490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4acdb-8001-49e9-a0fd-0245af55bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:31.719193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4acdb-8001-49e9-a0fd-0245af55bbec None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:31.719477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4acdb-8001-49e9-a0fd-0245af55bbec None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:31.719757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4acdb-8001-49e9-a0fd-0245af55bbec None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:31.720559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4acdb-8001-49e9-a0fd-0245af55bbec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:31.720872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4acdb-8001-49e9-a0fd-0245af55bbec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:31.722312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4acdb-8001-49e9-a0fd-0245af55bbec None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:31.732326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 479/954] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:31.735518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500bbd5-81f2-4947-b1da-06afeb4b35c5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:31.871104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500bbd5-81f2-4947-b1da-06afeb4b35c5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:31.881947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:31.910365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6072020-34af-44f3-8472-5d93ad54c319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:31.911079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6072020-34af-44f3-8472-5d93ad54c319 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:31.911366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6072020-34af-44f3-8472-5d93ad54c319 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:31.911641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6072020-34af-44f3-8472-5d93ad54c319 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:31.912353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6072020-34af-44f3-8472-5d93ad54c319 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:31.913811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6072020-34af-44f3-8472-5d93ad54c319 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:32.001782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dca7f-80d5-4412-89b8-dfa5447263d2 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:32.001960 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dca7f-80d5-4412-89b8-dfa5447263d2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:32.002285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dca7f-80d5-4412-89b8-dfa5447263d2 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:32.003865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 480/955] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:32.006143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 476/956] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:19:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:32.009538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023067-60b5-4b3d-af1a-18f778d463e6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:32.009831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023067-60b5-4b3d-af1a-18f778d463e6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:32.010103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023067-60b5-4b3d-af1a-18f778d463e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:32.031742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-41023067-60b5-4b3d-af1a-18f778d463e6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:32.033089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41023067-60b5-4b3d-af1a-18f778d463e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:32.120344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 481/957] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:32.405983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6072020-34af-44f3-8472-5d93ad54c319 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:32.529803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6072020-34af-44f3-8472-5d93ad54c319 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:32.566804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:32.567497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:32.567779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:32.568047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:32.568664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:32.570040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:32.580150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:32.581203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:32.582723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 477/958] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:32.587409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41023067-60b5-4b3d-af1a-18f778d463e6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:32.671630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:32.695887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41023067-60b5-4b3d-af1a-18f778d463e6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:32.739151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc67137-5164-4b49-82d0-5bba4d8ab0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:32.739758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc67137-5164-4b49-82d0-5bba4d8ab0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:32.740037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc67137-5164-4b49-82d0-5bba4d8ab0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:32.740337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc67137-5164-4b49-82d0-5bba4d8ab0b6 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:32.740973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc67137-5164-4b49-82d0-5bba4d8ab0b6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:32.741269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc67137-5164-4b49-82d0-5bba4d8ab0b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:32.742639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc67137-5164-4b49-82d0-5bba4d8ab0b6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:32.748364 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 482/959] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:32.755288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:32.786973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:32.873503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0340b8c-9c47-4a7e-a293-e970b187b0a0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:32.910490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed0de5-bd54-4267-b061-ad0e23ff1710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:32.911197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed0de5-bd54-4267-b061-ad0e23ff1710 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:32.911476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed0de5-bd54-4267-b061-ad0e23ff1710 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:32.911751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed0de5-bd54-4267-b061-ad0e23ff1710 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:32.912527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed0de5-bd54-4267-b061-ad0e23ff1710 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:32.912974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed0de5-bd54-4267-b061-ad0e23ff1710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:32.914353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed0de5-bd54-4267-b061-ad0e23ff1710 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:32.919842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 478/960] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:33.240744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c03c24-9e3b-4b43-b77b-5d83fee757b9 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:33.241109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c03c24-9e3b-4b43-b77b-5d83fee757b9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:33.241335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c03c24-9e3b-4b43-b77b-5d83fee757b9 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:33.242981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 483/961] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:33.248532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666f547-95d0-49b2-a659-624e61218a1c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:33.248820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666f547-95d0-49b2-a659-624e61218a1c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:33.249098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666f547-95d0-49b2-a659-624e61218a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:33.268723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1666f547-95d0-49b2-a659-624e61218a1c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:33.270257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1666f547-95d0-49b2-a659-624e61218a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:33.348347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 479/962] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:33.617817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c03c24-9e3b-4b43-b77b-5d83fee757b9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:33.730879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c03c24-9e3b-4b43-b77b-5d83fee757b9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:33.748215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a833-43c6-4825-8a3c-b07f315f27fa None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:33.748922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a833-43c6-4825-8a3c-b07f315f27fa None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:33.749205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a833-43c6-4825-8a3c-b07f315f27fa None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:33.755346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 480/963] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:33.757142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263692-6fc1-44cc-b09b-6fe2eaa442f0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:33.757503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263692-6fc1-44cc-b09b-6fe2eaa442f0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:33.757860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb263692-6fc1-44cc-b09b-6fe2eaa442f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:33.772110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:33.773115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:33.773564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:33.774080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:33.774909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:33.776559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:33.777642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-eb263692-6fc1-44cc-b09b-6fe2eaa442f0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:33.778994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-eb263692-6fc1-44cc-b09b-6fe2eaa442f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:33.779757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING keystone.server.flask.common [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 13:19:33.799364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:33.820366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.sql.core [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 13:19:33.820366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 13:19:33.820366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [parameters: {'user_id': '24faef2d20a0426289a3036badbc1b73', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 13:19:33.820366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81686) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application conn.query(q) Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result.read() Aug 24 13:19:33.829833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 13:19:33.831237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application next(self.gen) Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application yield resource Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application next(self.gen) Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise self.value Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application session.commit() Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 13:19:33.833010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.session.flush() Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application compat.raise_( Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 13:19:33.836431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 13:19:33.840754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application conn.query(q) Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application result.read() Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 13:19:33.844720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [parameters: {'user_id': '24faef2d20a0426289a3036badbc1b73', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:33.849154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 13:19:33.851560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 13:19:33.855923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 13:19:33.855923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 13:19:33.855923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 13:19:33.855923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 13:19:33.855923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 13:19:33.855923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:33.855923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 484/964] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 13:19:33 2022] POST /identity/v3/users => generated 160 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:33.857861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:33.864376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:33.877096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 481/965] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:33.980308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab525ce9-d56e-4da9-8291-6e6a16c7b44c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:34.019408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ff903-105e-4be4-bee2-80d47686ea93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:34.020606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ff903-105e-4be4-bee2-80d47686ea93 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.021089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ff903-105e-4be4-bee2-80d47686ea93 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.021558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ff903-105e-4be4-bee2-80d47686ea93 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.030138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ff903-105e-4be4-bee2-80d47686ea93 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:34.032750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ff903-105e-4be4-bee2-80d47686ea93 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-d37ff903-105e-4be4-bee2-80d47686ea93 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:34.045536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:34.046558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 13:19:34.047659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:34.050575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 485/966] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:34.057762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb263692-6fc1-44cc-b09b-6fe2eaa442f0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:34.123674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef6ecb-8fcc-42b2-a761-8446c64b4d56 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.124026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef6ecb-8fcc-42b2-a761-8446c64b4d56 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.124240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef6ecb-8fcc-42b2-a761-8446c64b4d56 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.130168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 486/967] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:34.135703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7c60b-31b5-4780-bc5d-d4ea940b9f1f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.135985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7c60b-31b5-4780-bc5d-d4ea940b9f1f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.136287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7c60b-31b5-4780-bc5d-d4ea940b9f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.155841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9da7c60b-31b5-4780-bc5d-d4ea940b9f1f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:34.158191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9da7c60b-31b5-4780-bc5d-d4ea940b9f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:34.182611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb263692-6fc1-44cc-b09b-6fe2eaa442f0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:34.227522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:34.228137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.228450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.228718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.229577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:34.230266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:34.232162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:34.232373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:34.250760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 487/968] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:34.250760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7c60b-31b5-4780-bc5d-d4ea940b9f1f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:34.279889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 482/969] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 13:19:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:34.372457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7c60b-31b5-4780-bc5d-d4ea940b9f1f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:34.417473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5eba-3586-4a18-afd1-c5b631f843e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:34.418044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5eba-3586-4a18-afd1-c5b631f843e2 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.418372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5eba-3586-4a18-afd1-c5b631f843e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.418594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5eba-3586-4a18-afd1-c5b631f843e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.432277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab5eba-3586-4a18-afd1-c5b631f843e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-0cab5eba-3586-4a18-afd1-c5b631f843e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:34.437492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:34.438497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:34.439353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 488/970] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 13:19:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:34.445198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:34.459274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:34.530584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5eba-3586-4a18-afd1-c5b631f843e2 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:34.566368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e522e4-88d5-4c55-97ed-05bbba60d344 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:34.596083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:34.610581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:34.611965 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.612485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.612957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.614167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:34.614742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:34.617336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:34.628280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 483/971] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 13:19:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:34.633784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:34.659633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5eba-3586-4a18-afd1-c5b631f843e2 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:34.700554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:34.701160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.701445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.701717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.702310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:34.704540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:34.714157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:34.715414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:34.716605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 489/972] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:34.722622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:34.733096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:34.765998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809acfb1-e5dc-4fcd-8aea-fc589a617af5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:34.803071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:34.803941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.804122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.804417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.820508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:34.821967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:34.840560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:34.841594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:34.842482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:34.842482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:34.842482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:34.842482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 484/973] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:34.849370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:34.884037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96dd16-77aa-40df-ba69-14dbb65a3004 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:34.905258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:34.925171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:34.925792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:34.926070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:34.926344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:34.926986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:34.927290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:34.928707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:34.939485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 490/974] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:34.940622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:34.970327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58b928-3246-4eb7-a013-7a17c35ca6f3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:34.994659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:35.006534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf5411-0b5e-4f79-a3fb-f263be8105b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:35.010081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf5411-0b5e-4f79-a3fb-f263be8105b6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.010081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf5411-0b5e-4f79-a3fb-f263be8105b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.010081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf5411-0b5e-4f79-a3fb-f263be8105b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.010081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf5411-0b5e-4f79-a3fb-f263be8105b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:35.010081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf5411-0b5e-4f79-a3fb-f263be8105b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:35.010081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf5411-0b5e-4f79-a3fb-f263be8105b6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:35.022580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 485/975] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:35.025560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a314f04-6f61-45d8-a51e-feabd45a557e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.025841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a314f04-6f61-45d8-a51e-feabd45a557e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.026156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a314f04-6f61-45d8-a51e-feabd45a557e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.027574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 486/976] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:35.028122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a314f04-6f61-45d8-a51e-feabd45a557e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:35.055714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc5aab-1fab-4995-b990-1221477ee3a7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:35.095059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b24728-eb75-452d-907e-5ac6d9fc3f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:35.095743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24728-eb75-452d-907e-5ac6d9fc3f44 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.096028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24728-eb75-452d-907e-5ac6d9fc3f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.096345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b24728-eb75-452d-907e-5ac6d9fc3f44 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.096989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b24728-eb75-452d-907e-5ac6d9fc3f44 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:35.097398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b24728-eb75-452d-907e-5ac6d9fc3f44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:35.098857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b24728-eb75-452d-907e-5ac6d9fc3f44 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:35.105269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 491/977] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:35.106523 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40dc98-184a-4352-a2d2-9dffbd5ca40d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.106796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40dc98-184a-4352-a2d2-9dffbd5ca40d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.107074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40dc98-184a-4352-a2d2-9dffbd5ca40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.127425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1b40dc98-184a-4352-a2d2-9dffbd5ca40d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:35.129249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1b40dc98-184a-4352-a2d2-9dffbd5ca40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:35.156739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a314f04-6f61-45d8-a51e-feabd45a557e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:35.177774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:35.194995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:35.195629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.195901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.196170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.205152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:35.206444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 492/978] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:35.207298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40dc98-184a-4352-a2d2-9dffbd5ca40d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:35.207421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:35.229737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:35.230740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 13:19:35.231692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:35.235061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 487/979] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:19:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:35.237712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:35.291379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:35.327862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40dc98-184a-4352-a2d2-9dffbd5ca40d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:35.358154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9144a6-f380-4b20-8824-26ed0d3221a6 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:35.367662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:35.368303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.368587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.368861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.369588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] RBAC: Adding query filter params (interface=admin, service_id=aa1d51a11d234802b31c28437dfebe01, region_id=RegionOne) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:35.370222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:35.371604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:35.386013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 493/980] 10.222.0.73 () {56 vars in 1249 bytes} [Wed Aug 24 13:19:35 2022] GET /identity/v3/endpoints?service_id=aa1d51a11d234802b31c28437dfebe01&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:35.387072 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:35.402534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:35.403111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.403369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.403645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.404414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:35.404720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:35.406233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:35.427494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 488/981] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 13:19:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:35.436168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:35.445954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:35.511655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521baaa-f137-4653-84d1-c2047b378051 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:35.548080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f44cb-ac9c-4438-a6cc-621de9e2d22e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:35.548576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b2a66-14a6-40e2-911b-4d8e549485b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:35.549009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b2a66-14a6-40e2-911b-4d8e549485b7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.549291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b2a66-14a6-40e2-911b-4d8e549485b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.549563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b2a66-14a6-40e2-911b-4d8e549485b7 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.550066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:35.550214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b2a66-14a6-40e2-911b-4d8e549485b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:35.551605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b2a66-14a6-40e2-911b-4d8e549485b7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:35.557978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 494/982] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:19:35 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:19:35.558903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b2a66-14a6-40e2-911b-4d8e549485b7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:35.589597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd9a99-1e40-4875-8402-16e499d93938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:35.590530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd9a99-1e40-4875-8402-16e499d93938 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.590975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd9a99-1e40-4875-8402-16e499d93938 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.591395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd9a99-1e40-4875-8402-16e499d93938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.694338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd9a99-1e40-4875-8402-16e499d93938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:35.695849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd9a99-1e40-4875-8402-16e499d93938 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:35.696069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b2a66-14a6-40e2-911b-4d8e549485b7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:35.700645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:35.747753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dac34c-0de9-4f57-8119-7d7016d7bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:35.747989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac34c-0de9-4f57-8119-7d7016d7bf1a None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:35.748199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac34c-0de9-4f57-8119-7d7016d7bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:35.748864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dac34c-0de9-4f57-8119-7d7016d7bf1a None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:35.750594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dac34c-0de9-4f57-8119-7d7016d7bf1a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:35.753231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dac34c-0de9-4f57-8119-7d7016d7bf1a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:35.824112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 489/983] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:35 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 13:19:35.830636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd9a99-1e40-4875-8402-16e499d93938 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:35.835899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:35.836792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 495/984] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:19:35 2022] POST /identity/v3/services => generated 242 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:36.018470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd9a99-1e40-4875-8402-16e499d93938 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:36.070360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:36.085956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:36.085956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:36.085956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:36.087183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:36.088705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:36.106064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:36.107101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:36.108652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:36.108652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:36.108652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:36.108652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 490/985] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:36.113189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dac34c-0de9-4f57-8119-7d7016d7bf1a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:36.269224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dac34c-0de9-4f57-8119-7d7016d7bf1a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:36.322177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:36.322922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:36.323261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:36.323530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:36.324447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:36.324770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:36.326316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:36.341319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 496/986] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:36.347516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:36.429662 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:36.491660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29dea0-37f8-4bc1-bf1d-6caa89204f34 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:36.537328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:36.538171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:36.538462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:36.538899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:36.551697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:36.553883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:36.571977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:36.573037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:36.573929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:36.573929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:36.573929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:36.573929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:36.573929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:36.573929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:36.573929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:36.573929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:36.580195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 491/987] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:36.581778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:36.609791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:36.707548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07015d-3b6f-467f-a7bb-1bbdbb0ce5d7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:36.750149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf09fc1-b02d-436b-8ee8-e37f2e097a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:36.750938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf09fc1-b02d-436b-8ee8-e37f2e097a4c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:36.751522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf09fc1-b02d-436b-8ee8-e37f2e097a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:36.751522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf09fc1-b02d-436b-8ee8-e37f2e097a4c None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:36.752266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf09fc1-b02d-436b-8ee8-e37f2e097a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:36.752571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf09fc1-b02d-436b-8ee8-e37f2e097a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:36.753977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf09fc1-b02d-436b-8ee8-e37f2e097a4c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:36.767456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 497/988] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:36.773777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:36.900394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4458e-e304-4447-b841-f9dcf524d810 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:36.912858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:36.939237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb1c35-e4e4-4299-8164-2392175e2c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:36.939906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb1c35-e4e4-4299-8164-2392175e2c4c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:36.940201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb1c35-e4e4-4299-8164-2392175e2c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:36.940504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb1c35-e4e4-4299-8164-2392175e2c4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:36.941239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb1c35-e4e4-4299-8164-2392175e2c4c None admin] RBAC: Adding query filter params (role.id=2a934a9192c84ef3a65ed7acfc442b2f, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=b56fb09428cb4ab2b49ca199058a5a7a) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:36.941547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb1c35-e4e4-4299-8164-2392175e2c4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:36.942976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb1c35-e4e4-4299-8164-2392175e2c4c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:36.956210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 492/989] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:36 2022] GET /identity/v3/role_assignments?user.id=b56fb09428cb4ab2b49ca199058a5a7a&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=2a934a9192c84ef3a65ed7acfc442b2f => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:37.647285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b6b8-689d-4850-bba1-dc3673111a0c None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:37.647682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b6b8-689d-4850-bba1-dc3673111a0c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:37.647865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b6b8-689d-4850-bba1-dc3673111a0c None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:37.649278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 498/990] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:37.654586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674faca-649d-4ff0-9b5b-54846d1992d5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:37.654885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674faca-649d-4ff0-9b5b-54846d1992d5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:37.655165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674faca-649d-4ff0-9b5b-54846d1992d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:37.675037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8674faca-649d-4ff0-9b5b-54846d1992d5 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:37.676353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8674faca-649d-4ff0-9b5b-54846d1992d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:37.759616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 493/991] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:37.931055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696ec83-3c5a-422c-9581-033fdaee38ad None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:37.931546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696ec83-3c5a-422c-9581-033fdaee38ad None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:37.931546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696ec83-3c5a-422c-9581-033fdaee38ad None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:37.933202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 499/992] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:37.938532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0a15a-8f4b-4bd3-91a3-a55963e2d300 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:37.938988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0a15a-8f4b-4bd3-91a3-a55963e2d300 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:37.939191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0a15a-8f4b-4bd3-91a3-a55963e2d300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:37.958437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1fa0a15a-8f4b-4bd3-91a3-a55963e2d300 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:37.959630 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1fa0a15a-8f4b-4bd3-91a3-a55963e2d300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:38.031716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 494/993] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:38.032961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696ec83-3c5a-422c-9581-033fdaee38ad None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:38.149844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:38.167663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696ec83-3c5a-422c-9581-033fdaee38ad None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:38.203810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:38.204394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.204853 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.205140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.205729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:38.207106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:38.216373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:38.217418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:38.218247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 500/994] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 13:19:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:38.224474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0a15a-8f4b-4bd3-91a3-a55963e2d300 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:38.261996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:38.334794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0a15a-8f4b-4bd3-91a3-a55963e2d300 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:38.345386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:38.376240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:38.377079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.377366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.377642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.378307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:38.378619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:38.380050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:38.390942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 495/995] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:38.395435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:38.467204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2826b0-1be7-4b1d-8f09-ca2a7a7b5559 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:38.505428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:38.505888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.506223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.506372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.506921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:38.508417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:38.516311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:38.519387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:38.520406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:38.521452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 501/996] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:38.525235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353589f-8cfb-4bef-a5d7-73d053ed65cb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:38.526568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:38.545425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:38.564045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:38.564737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.565015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.565284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.565917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:38.566217 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:38.567574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:38.578034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 496/997] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 13:19:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:38.578925 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:38.642089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca7b1f-2eba-40ba-a2fc-be5835a6ca91 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:38.678524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:38.679123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.679398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.679669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.680333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] RBAC: Adding query filter params (name=compute) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:38.680667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:38.682032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:38.687721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 502/998] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:38.693381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:38.695071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4d1df-c75c-4883-917d-9dbb46a772fa None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:38.735624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac4db7-5f9e-4faa-84c6-de500c3a4acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:38.736158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac4db7-5f9e-4faa-84c6-de500c3a4acf None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.736474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac4db7-5f9e-4faa-84c6-de500c3a4acf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.736781 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac4db7-5f9e-4faa-84c6-de500c3a4acf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.737648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac4db7-5f9e-4faa-84c6-de500c3a4acf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:38.739039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac4db7-5f9e-4faa-84c6-de500c3a4acf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:38.779147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:38.804002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf8a1a-4301-4219-ba3a-7cf8046998aa None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:38.839175 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbc98c-77a3-4492-9b91-fc3427933cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:38.839788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbc98c-77a3-4492-9b91-fc3427933cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.840071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbc98c-77a3-4492-9b91-fc3427933cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.840376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbc98c-77a3-4492-9b91-fc3427933cf0 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.841017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbc98c-77a3-4492-9b91-fc3427933cf0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:38.841323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbc98c-77a3-4492-9b91-fc3427933cf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:38.852616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 497/999] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:19:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:38.852848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdbc98c-77a3-4492-9b91-fc3427933cf0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:38.857672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 503/1000] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:38.864542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac4db7-5f9e-4faa-84c6-de500c3a4acf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:38.894559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:38.947479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11144fa-5ce4-4a56-9393-91420274af01 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.948027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11144fa-5ce4-4a56-9393-91420274af01 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.949062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11144fa-5ce4-4a56-9393-91420274af01 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.950886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 504/1001] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:38.957330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e327d7-b28f-43a4-9d80-623b510e0895 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:38.957625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e327d7-b28f-43a4-9d80-623b510e0895 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:38.957880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e327d7-b28f-43a4-9d80-623b510e0895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:38.979828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-84e327d7-b28f-43a4-9d80-623b510e0895 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:38.981093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-84e327d7-b28f-43a4-9d80-623b510e0895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:38.990901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac4db7-5f9e-4faa-84c6-de500c3a4acf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:39.033662 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:39.034281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:39.034561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:39.034829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:39.035537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] RBAC: Adding query filter params (interface=public, service_id=0188d3c3be2c4d468855e247d03ebf3a, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:39.035844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:39.037276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:39.043925 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 498/1002] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 13:19:38 2022] GET /identity/v3/endpoints?service_id=0188d3c3be2c4d468855e247d03ebf3a&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:39.050818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:39.064917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 505/1003] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:39.144343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:39.163005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a5c48-a63a-4b9b-9d52-65e62d3512cd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:39.199453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750cd82-0ea7-4096-9312-c38d95ddd8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:39.200064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750cd82-0ea7-4096-9312-c38d95ddd8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:39.200381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750cd82-0ea7-4096-9312-c38d95ddd8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:39.200656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750cd82-0ea7-4096-9312-c38d95ddd8f5 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:39.201323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750cd82-0ea7-4096-9312-c38d95ddd8f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:39.203247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750cd82-0ea7-4096-9312-c38d95ddd8f5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:39.210587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 499/1004] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:19:39 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:19:39.340603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e327d7-b28f-43a4-9d80-623b510e0895 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:39.447548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:39.480728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e327d7-b28f-43a4-9d80-623b510e0895 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:39.522679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:39.523385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:39.523674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:39.523952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] PATH_INFO: `/v3/services/image` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:39.524596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:39.525981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:39.537377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:39.538475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:39.539416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:39.540842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 506/1005] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 13:19:39 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:39.544839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750cd82-0ea7-4096-9312-c38d95ddd8f5 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:39.650782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750cd82-0ea7-4096-9312-c38d95ddd8f5 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:39.685821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae12a00-764c-4f37-8cc0-77bc60465b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:39.688560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae12a00-764c-4f37-8cc0-77bc60465b75 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:39.688560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae12a00-764c-4f37-8cc0-77bc60465b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:39.688560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae12a00-764c-4f37-8cc0-77bc60465b75 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:39.688560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae12a00-764c-4f37-8cc0-77bc60465b75 None admin] RBAC: Adding query filter params (name=image) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:39.688560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae12a00-764c-4f37-8cc0-77bc60465b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:39.689805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae12a00-764c-4f37-8cc0-77bc60465b75 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:39.695407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 500/1006] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:39.701482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:39.821341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772dbb0-cb4a-4194-b43e-35ada552af01 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:39.867010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d21777-cd68-4f6f-aca3-85d2ab772c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:39.867246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d21777-cd68-4f6f-aca3-85d2ab772c1f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:39.867507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d21777-cd68-4f6f-aca3-85d2ab772c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:39.867825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d21777-cd68-4f6f-aca3-85d2ab772c1f None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:39.868433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d21777-cd68-4f6f-aca3-85d2ab772c1f None admin] RBAC: Adding query filter params (type=image) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:39.868761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d21777-cd68-4f6f-aca3-85d2ab772c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:39.870165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d21777-cd68-4f6f-aca3-85d2ab772c1f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:39.871559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:39.884228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 507/1007] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:40.944033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2b94f-e25f-4b11-bf30-0d02291e801b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:40.944574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2b94f-e25f-4b11-bf30-0d02291e801b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:40.944652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2b94f-e25f-4b11-bf30-0d02291e801b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:40.945958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 501/1008] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:40.951686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1ed1-d036-4213-b407-7924d8b2c378 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:40.952126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1ed1-d036-4213-b407-7924d8b2c378 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:40.952356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1ed1-d036-4213-b407-7924d8b2c378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:40.973465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-dd9d1ed1-d036-4213-b407-7924d8b2c378 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:40.974688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dd9d1ed1-d036-4213-b407-7924d8b2c378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:41.047626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 508/1009] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:41.208393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60227e56-aa9a-481c-9cba-02fdb86c9cac None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:41.208681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60227e56-aa9a-481c-9cba-02fdb86c9cac None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:41.209136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60227e56-aa9a-481c-9cba-02fdb86c9cac None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:41.210486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 502/1010] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:41.216408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba8444-ff26-454d-bec8-c41c2da4c45b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:41.216701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba8444-ff26-454d-bec8-c41c2da4c45b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:41.216980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba8444-ff26-454d-bec8-c41c2da4c45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:41.236561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3aba8444-ff26-454d-bec8-c41c2da4c45b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:41.237760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3aba8444-ff26-454d-bec8-c41c2da4c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:41.314025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 509/1011] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:41.323165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60227e56-aa9a-481c-9cba-02fdb86c9cac None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:41.337629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:41.440982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60227e56-aa9a-481c-9cba-02fdb86c9cac None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:41.476074 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c3bc0-2383-4507-82fc-ed5d81279655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:41.476956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c3bc0-2383-4507-82fc-ed5d81279655 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:41.477246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c3bc0-2383-4507-82fc-ed5d81279655 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:41.477575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c3bc0-2383-4507-82fc-ed5d81279655 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:41.478704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c3bc0-2383-4507-82fc-ed5d81279655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:41.480146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c3bc0-2383-4507-82fc-ed5d81279655 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:41.487526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 503/1012] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:41.489126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba8444-ff26-454d-bec8-c41c2da4c45b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:41.557691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:41.587050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c3bc0-2383-4507-82fc-ed5d81279655 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:41.603313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba8444-ff26-454d-bec8-c41c2da4c45b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:41.642381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290189bb-25f4-46d4-873d-d20aa5d9a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:41.643274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290189bb-25f4-46d4-873d-d20aa5d9a4bf None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:41.643559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290189bb-25f4-46d4-873d-d20aa5d9a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:41.643837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290189bb-25f4-46d4-873d-d20aa5d9a4bf None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:41.644714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290189bb-25f4-46d4-873d-d20aa5d9a4bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:41.646491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290189bb-25f4-46d4-873d-d20aa5d9a4bf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:41.677239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:41.699349 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c3bc0-2383-4507-82fc-ed5d81279655 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:41.732842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 510/1013] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:19:41 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:41.742141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7336b-5871-4d8a-812b-c9ab1131a029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:41.742746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7336b-5871-4d8a-812b-c9ab1131a029 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:41.743033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7336b-5871-4d8a-812b-c9ab1131a029 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:41.743314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7336b-5871-4d8a-812b-c9ab1131a029 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:41.746880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7336b-5871-4d8a-812b-c9ab1131a029 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:41.746880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7336b-5871-4d8a-812b-c9ab1131a029 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5ad7336b-5871-4d8a-812b-c9ab1131a029 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:41.759153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:41.760523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 13:19:41.761520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:41.764586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 504/1014] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:41.766920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290189bb-25f4-46d4-873d-d20aa5d9a4bf None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:41.811919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f68c15-b2cd-404b-88c3-aaabffee9fa5 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:41.812381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f68c15-b2cd-404b-88c3-aaabffee9fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:41.812658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f68c15-b2cd-404b-88c3-aaabffee9fa5 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:41.814406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 505/1015] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:41.820009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85a81-59ac-4e6d-ba7f-7bc9d4d00255 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:41.820394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85a81-59ac-4e6d-ba7f-7bc9d4d00255 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:41.820624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc85a81-59ac-4e6d-ba7f-7bc9d4d00255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:41.842414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6bc85a81-59ac-4e6d-ba7f-7bc9d4d00255 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:41.844078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6bc85a81-59ac-4e6d-ba7f-7bc9d4d00255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:41.898717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290189bb-25f4-46d4-873d-d20aa5d9a4bf None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:41.911043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:41.927465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 506/1016] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:41.942443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:41.943121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:41.943402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:41.943701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:41.944436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:41.944743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:41.946120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:41.951539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 511/1017] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:41.958197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc85a81-59ac-4e6d-ba7f-7bc9d4d00255 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:42.034093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:42.064503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc85a81-59ac-4e6d-ba7f-7bc9d4d00255 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:42.101211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:42.101745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:42.102028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:42.102301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:42.102940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:42.103243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:42.104645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:42.111861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 507/1018] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 13:19:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:42.119506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:42.199204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:42.261962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12861ba3-0918-4b79-a479-e2d4ed97a370 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:42.306283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99e6c5-61cb-478d-a2a6-a60e2294b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:42.306889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e6c5-61cb-478d-a2a6-a60e2294b77a None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:42.307170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e6c5-61cb-478d-a2a6-a60e2294b77a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:42.307440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e6c5-61cb-478d-a2a6-a60e2294b77a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:42.308108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99e6c5-61cb-478d-a2a6-a60e2294b77a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:42.309511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99e6c5-61cb-478d-a2a6-a60e2294b77a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:42.341831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0ddf2-c23c-4e34-a1d0-3fe4a091d7ea None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:42.393720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 512/1019] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:19:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:19:42.411170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:42.416787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca12603-3a0a-4484-954e-76798e459aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:42.417485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12603-3a0a-4484-954e-76798e459aec None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:42.417784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12603-3a0a-4484-954e-76798e459aec None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:42.418045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca12603-3a0a-4484-954e-76798e459aec None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:42.418726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca12603-3a0a-4484-954e-76798e459aec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:42.420148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca12603-3a0a-4484-954e-76798e459aec None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:42.542391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 508/1020] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:19:42 2022] POST /identity/v3/services => generated 242 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:19:43.833993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197b65-7ff6-48aa-9d5a-4bcdc3824e27 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:43.834497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197b65-7ff6-48aa-9d5a-4bcdc3824e27 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:43.834576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197b65-7ff6-48aa-9d5a-4bcdc3824e27 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:43.836145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 513/1021] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:43.841647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caee024-8b74-4c55-993b-e948374b648f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:43.842082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caee024-8b74-4c55-993b-e948374b648f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:43.842270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caee024-8b74-4c55-993b-e948374b648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:43.861604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0caee024-8b74-4c55-993b-e948374b648f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:43.862813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0caee024-8b74-4c55-993b-e948374b648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:43.944327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 509/1022] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:44.215097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2197b65-7ff6-48aa-9d5a-4bcdc3824e27 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:44.251366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:44.329868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2197b65-7ff6-48aa-9d5a-4bcdc3824e27 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:44.366675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98634886-d3e8-4ff9-96c6-ec28724b77c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:44.367315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98634886-d3e8-4ff9-96c6-ec28724b77c7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:44.367597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98634886-d3e8-4ff9-96c6-ec28724b77c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:44.367873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98634886-d3e8-4ff9-96c6-ec28724b77c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:44.369058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98634886-d3e8-4ff9-96c6-ec28724b77c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:44.370560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98634886-d3e8-4ff9-96c6-ec28724b77c7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:44.372392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 514/1023] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:44.378933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caee024-8b74-4c55-993b-e948374b648f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:44.479810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efbd73-a1e6-4e53-a926-ce4b03945ee8 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:44.479989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efbd73-a1e6-4e53-a926-ce4b03945ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:44.480299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efbd73-a1e6-4e53-a926-ce4b03945ee8 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:44.482539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 515/1024] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:44.484802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d678212-76fc-413c-918d-63ba10bd9d0d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:44.485086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d678212-76fc-413c-918d-63ba10bd9d0d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:44.485358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d678212-76fc-413c-918d-63ba10bd9d0d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:44.487187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 516/1025] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:44.488520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd8ec4-d15a-4b1a-9277-0a0f6b4df791 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:44.488811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd8ec4-d15a-4b1a-9277-0a0f6b4df791 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:44.489087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd8ec4-d15a-4b1a-9277-0a0f6b4df791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:44.496817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caee024-8b74-4c55-993b-e948374b648f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:44.507805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-26dd8ec4-d15a-4b1a-9277-0a0f6b4df791 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:44.509030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-26dd8ec4-d15a-4b1a-9277-0a0f6b4df791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:44.514842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:44.533097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ab91c-1f32-40a0-a998-c13defcf21a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:44.536527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab91c-1f32-40a0-a998-c13defcf21a1 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:44.536527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab91c-1f32-40a0-a998-c13defcf21a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:44.536527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ab91c-1f32-40a0-a998-c13defcf21a1 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:44.536527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ab91c-1f32-40a0-a998-c13defcf21a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:44.536527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ab91c-1f32-40a0-a998-c13defcf21a1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:44.580646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 517/1026] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:44.581914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52f23-942e-4d6d-a3a2-85342fb9168d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:44.582202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52f23-942e-4d6d-a3a2-85342fb9168d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:44.582476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52f23-942e-4d6d-a3a2-85342fb9168d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:44.601779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-73c52f23-942e-4d6d-a3a2-85342fb9168d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:44.603174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-73c52f23-942e-4d6d-a3a2-85342fb9168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:44.626705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:44.646232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 510/1027] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:19:44 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:44.681122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 518/1028] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:44.855075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ab91c-1f32-40a0-a998-c13defcf21a1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:44.948803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c52f23-942e-4d6d-a3a2-85342fb9168d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:44.971942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ab91c-1f32-40a0-a998-c13defcf21a1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.018483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01acb-293e-4cd1-9a24-9e2baf4d7178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.019044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01acb-293e-4cd1-9a24-9e2baf4d7178 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.019321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01acb-293e-4cd1-9a24-9e2baf4d7178 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.019595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01acb-293e-4cd1-9a24-9e2baf4d7178 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.031604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc01acb-293e-4cd1-9a24-9e2baf4d7178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1cc01acb-293e-4cd1-9a24-9e2baf4d7178 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:45.036395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:45.037366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:45.038241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 511/1029] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 13:19:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:45.043699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01acb-293e-4cd1-9a24-9e2baf4d7178 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.058374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c52f23-942e-4d6d-a3a2-85342fb9168d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.084029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:45.098153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.099269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.100008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.100008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] PATH_INFO: `/v3/services/image` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.100775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.102440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:45.112831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:45.113867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:45.114703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:45.118963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 519/1030] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 13:19:44 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:45.119660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.146496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:45.157973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01acb-293e-4cd1-9a24-9e2baf4d7178 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.196084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.196833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.197138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.197423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.198278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:45.198391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.199998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.210129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 512/1031] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:45.216097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.243057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218e18c-4e01-46bc-be06-091bb789a523 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.284040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.284689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.284974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.285251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.285893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] RBAC: Adding query filter params (name=image) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:45.286212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.287611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.293339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 520/1032] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:45.299687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.331846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:45.338293 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9997f-25ce-4f78-ad75-10a31b03abcd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.376376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.377023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.377303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.377573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.395878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.397916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.411402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:45.415145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:45.420739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:45.422238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:45.422238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:45.422238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:45.422238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 513/1033] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:45.424266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.438333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f5f56-d265-49d7-a6ce-e5bab0337087 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.480465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.481131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.481447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.481744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.482354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:45.482730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.484982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.490975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 521/1034] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:45.496498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.543505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887f561-1824-4b94-b7d0-4153497dc67f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.584721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.585323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.585610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.585868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.586791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:45.586791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.588246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.606214 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 514/1035] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:45.612653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.623943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:45.649047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa387-1599-4f37-9cd1-d42d4f622ea3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.692642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.693236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.693522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.693935 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.694505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] RBAC: Adding query filter params (interface=public, service_id=17cea299983645f2a5bb34f10ddfda3b, region_id=RegionOne) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:45.694816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.696218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.707824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 522/1036] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/endpoints?service_id=17cea299983645f2a5bb34f10ddfda3b&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:45.709083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.728742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:45.744259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9959ce4-e106-479d-9779-1fecb0d8b26c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.794761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.795095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.795288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.795567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.804270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.805718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:45.822412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:45.830777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:45.831736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 515/1037] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:45.833868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:45.873850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722aeb-c4fd-4c94-970d-6db1d1dfaece None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:45.918271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b24ca-fa2b-4a03-9bdb-55db25233c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:45.918897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b24ca-fa2b-4a03-9bdb-55db25233c73 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:45.919182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b24ca-fa2b-4a03-9bdb-55db25233c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:45.919458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b24ca-fa2b-4a03-9bdb-55db25233c73 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:45.920800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b24ca-fa2b-4a03-9bdb-55db25233c73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:45.922735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b24ca-fa2b-4a03-9bdb-55db25233c73 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:45.930821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 523/1038] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/services => generated 1957 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:19:45.938861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:45.984509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f71497-fb54-4a1c-ba68-df862ac6be18 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:46.026860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:46.027496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:46.027773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:46.028043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:46.028926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:46.029239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:46.031353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:46.056315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 516/1039] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:19:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:46.059411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b24ca-fa2b-4a03-9bdb-55db25233c73 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:46.157109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:46.182070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b24ca-fa2b-4a03-9bdb-55db25233c73 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:46.222228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449cc9b-5666-4f3c-9ddb-9e6cc5880376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:46.223215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449cc9b-5666-4f3c-9ddb-9e6cc5880376 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:46.223703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449cc9b-5666-4f3c-9ddb-9e6cc5880376 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:46.224291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449cc9b-5666-4f3c-9ddb-9e6cc5880376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:46.297781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449cc9b-5666-4f3c-9ddb-9e6cc5880376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:46.299950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449cc9b-5666-4f3c-9ddb-9e6cc5880376 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:46.383723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:46.417009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 524/1040] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:19:46.418279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:46.525466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9aff8-92cb-486f-af9d-cdcd63fc8821 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:46.556825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:46.565861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb32c2-32a6-416b-8433-40fb531b0c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:46.566492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb32c2-32a6-416b-8433-40fb531b0c68 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:46.566762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb32c2-32a6-416b-8433-40fb531b0c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:46.572419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb32c2-32a6-416b-8433-40fb531b0c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:46.577512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb32c2-32a6-416b-8433-40fb531b0c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:46.579033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb32c2-32a6-416b-8433-40fb531b0c68 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-facb32c2-32a6-416b-8433-40fb531b0c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:46.594364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:46.595517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:46.596504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:46.596504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:46.596504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:46.607238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 517/1041] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:46.610340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449cc9b-5666-4f3c-9ddb-9e6cc5880376 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:46.721522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449cc9b-5666-4f3c-9ddb-9e6cc5880376 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:46.733363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e13101-79bd-4647-8d78-c62a36df28bb None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:46.734185 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e13101-79bd-4647-8d78-c62a36df28bb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:46.734464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e13101-79bd-4647-8d78-c62a36df28bb None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:46.736088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 518/1042] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:46.741173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26f973-3748-4834-b437-e656a425c445 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:46.741716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26f973-3748-4834-b437-e656a425c445 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:46.741716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26f973-3748-4834-b437-e656a425c445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:46.760595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5f26f973-3748-4834-b437-e656a425c445 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:46.761817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5f26f973-3748-4834-b437-e656a425c445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:46.762650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:46.763259 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:46.763540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:46.763810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:46.764566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:46.764868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:46.766355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:46.779900 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 525/1043] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:46.785807 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:46.834843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:46.848348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 519/1044] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:46.901821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940d1b4-07e6-4c23-90b4-cf5a2cb42a1c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:46.941835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:46.942802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:46.943236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:46.943662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:46.951220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:46.954219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:46.966423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:46.967485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:46.969742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:46.969742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:46.969742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:46.969742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:46.969742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:46.969742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:46.969742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:46.969742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:46.972381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 526/1045] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:46.975139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26f973-3748-4834-b437-e656a425c445 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:47.091081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26f973-3748-4834-b437-e656a425c445 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:47.102675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:47.118537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:47.129527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:47.130126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:47.130405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:47.130682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:47.131370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:47.131674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:47.133199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:47.152339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 520/1046] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:47.153966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:47.238037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:47.241958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc20b-8117-4a46-96de-8bf0b2cdb1c1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:47.268370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f1c13-4a9a-4526-8ecb-9b0f21937935 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:47.277971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cb5e3-345c-4ecd-a830-9a4a8fdd649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:47.279094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cb5e3-345c-4ecd-a830-9a4a8fdd649e None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:47.279547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cb5e3-345c-4ecd-a830-9a4a8fdd649e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:47.280136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969cb5e3-345c-4ecd-a830-9a4a8fdd649e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:47.292228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969cb5e3-345c-4ecd-a830-9a4a8fdd649e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-969cb5e3-345c-4ecd-a830-9a4a8fdd649e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:47.298432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:47.299412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:47.300308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 527/1047] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:47.305525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cb5e3-345c-4ecd-a830-9a4a8fdd649e None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:47.315204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:47.316077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:47.316419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:47.316572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:47.317372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] RBAC: Adding query filter params (role.id=2a934a9192c84ef3a65ed7acfc442b2f, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=40561a306123461d92bc669be3962db3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:47.318141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:47.320174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:47.340209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 521/1048] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:47 2022] GET /identity/v3/role_assignments?user.id=40561a306123461d92bc669be3962db3&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=2a934a9192c84ef3a65ed7acfc442b2f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:47.427032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969cb5e3-345c-4ecd-a830-9a4a8fdd649e None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:47.469557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:47.470184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:47.470459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:47.470728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:47.471406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:47.471711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:47.474154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:47.483990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 528/1049] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:47.489851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:47.523411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:47.600149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654e54d-ab48-4c5f-8757-e21484e85dcb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:47.635822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0221da-65f2-4ead-8d89-a6a845614739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:47.636415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0221da-65f2-4ead-8d89-a6a845614739 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:47.636697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0221da-65f2-4ead-8d89-a6a845614739 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:47.636969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0221da-65f2-4ead-8d89-a6a845614739 None admin] PATH_INFO: `/v3/users/ae7c53111fac47aaa0bb4d380f523101` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:47.652213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0221da-65f2-4ead-8d89-a6a845614739 None admin] RBAC: Authorizing `identity:get_user(user_id=ae7c53111fac47aaa0bb4d380f523101)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:47.653657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0221da-65f2-4ead-8d89-a6a845614739 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:47.656132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 522/1050] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:19:47 2022] GET /identity/v3/users/ae7c53111fac47aaa0bb4d380f523101 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:47.661744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:47.684451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:47.770940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a039e3-a11d-427b-8c50-f0ad86fc8310 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:47.807983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238954e-4978-4e20-a9b5-26aa61a4a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:47.808678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238954e-4978-4e20-a9b5-26aa61a4a2af None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:47.808964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238954e-4978-4e20-a9b5-26aa61a4a2af None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:47.809233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238954e-4978-4e20-a9b5-26aa61a4a2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:47.880084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4394-0866-47d7-96de-f1317923e8cb None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:47.880398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4394-0866-47d7-96de-f1317923e8cb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:47.880675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4394-0866-47d7-96de-f1317923e8cb None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:47.882210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 523/1051] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:47.887339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfb71d-a3d8-490a-89b8-9f517925f6ab None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:47.887625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfb71d-a3d8-490a-89b8-9f517925f6ab None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:47.887899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfb71d-a3d8-490a-89b8-9f517925f6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:47.894961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238954e-4978-4e20-a9b5-26aa61a4a2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:47.896464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238954e-4978-4e20-a9b5-26aa61a4a2af None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:47.907094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-38cfb71d-a3d8-490a-89b8-9f517925f6ab None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:47.908653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-38cfb71d-a3d8-490a-89b8-9f517925f6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:47.919780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:48.000403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 524/1052] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:48.017378 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 529/1053] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:47 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:19:48.023257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfb71d-a3d8-490a-89b8-9f517925f6ab None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:48.045739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:48.126496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfb71d-a3d8-490a-89b8-9f517925f6ab None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:48.185391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346715e-12a3-4881-9dc7-b4338c698faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:48.186032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346715e-12a3-4881-9dc7-b4338c698faa None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:48.186312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346715e-12a3-4881-9dc7-b4338c698faa None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:48.186617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346715e-12a3-4881-9dc7-b4338c698faa None admin] PATH_INFO: `/v3/projects/61e1e75e0707484f999804a7d9404c51` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:48.188221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346715e-12a3-4881-9dc7-b4338c698faa None admin] RBAC: Authorizing `identity:get_project(project_id=61e1e75e0707484f999804a7d9404c51)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:48.189663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346715e-12a3-4881-9dc7-b4338c698faa None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:48.191535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 525/1054] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:19:48 2022] GET /identity/v3/projects/61e1e75e0707484f999804a7d9404c51 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:48.197464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238954e-4978-4e20-a9b5-26aa61a4a2af None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:48.270976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346715e-12a3-4881-9dc7-b4338c698faa None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:48.312845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238954e-4978-4e20-a9b5-26aa61a4a2af None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:48.337099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:48.355446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:48.356053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:48.356479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:48.356757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:48.357496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=61e1e75e0707484f999804a7d9404c51, user.id=ae7c53111fac47aaa0bb4d380f523101) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:48.357799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:48.359488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:48.365991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 530/1055] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:48 2022] GET /identity/v3/role_assignments?user.id=ae7c53111fac47aaa0bb4d380f523101&scope.project.id=61e1e75e0707484f999804a7d9404c51&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:48.374839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:48.391972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346715e-12a3-4881-9dc7-b4338c698faa None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:48.439798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:48.440626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:48.440918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:48.441139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:48.446917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:48.446917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:48.463334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:48.464742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 13:19:48.465620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:48.468855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 526/1056] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 13:19:48 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:48.475862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:48.599261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10f2aa-19dc-4b7b-ab66-f939fbb604a7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:48.634532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:48.635154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:48.635435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:48.635709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:48.636381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:48.636685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:48.638057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:48.648410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 531/1057] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:48.649871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:48.776349 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805744-f6aa-4299-9aea-c644d43796d3 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:48.807073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67e0f7-5ccd-47a7-aeb4-5b7910043ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:48.807782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67e0f7-5ccd-47a7-aeb4-5b7910043ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:48.808139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67e0f7-5ccd-47a7-aeb4-5b7910043ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:48.808512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67e0f7-5ccd-47a7-aeb4-5b7910043ae5 None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:48.809230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67e0f7-5ccd-47a7-aeb4-5b7910043ae5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:48.809609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67e0f7-5ccd-47a7-aeb4-5b7910043ae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:48.811075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67e0f7-5ccd-47a7-aeb4-5b7910043ae5 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:48.833529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 527/1058] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:48.833728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:48.888795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:48.945823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74bf3c-b368-4fec-9f14-7db2079ec2a5 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:48.985111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c14cf-d72e-4dd3-8f24-42dc98667adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:48.985740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c14cf-d72e-4dd3-8f24-42dc98667adf None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:48.986016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c14cf-d72e-4dd3-8f24-42dc98667adf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:48.986287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c14cf-d72e-4dd3-8f24-42dc98667adf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:48.986974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c14cf-d72e-4dd3-8f24-42dc98667adf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:48.988426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c14cf-d72e-4dd3-8f24-42dc98667adf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:49.076765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 532/1059] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:19:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:19:49.333134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd4c5e-3317-492c-b20e-f0c19580c5a0 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:49.333657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd4c5e-3317-492c-b20e-f0c19580c5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:49.333737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd4c5e-3317-492c-b20e-f0c19580c5a0 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:49.335218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 528/1060] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:49.340559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15bb7a-f117-4d5f-aaa8-4bc0a6d21e8c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:49.340975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15bb7a-f117-4d5f-aaa8-4bc0a6d21e8c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:49.341146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15bb7a-f117-4d5f-aaa8-4bc0a6d21e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:49.359909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2e15bb7a-f117-4d5f-aaa8-4bc0a6d21e8c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:49.361233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2e15bb7a-f117-4d5f-aaa8-4bc0a6d21e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:49.445920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 533/1061] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:49.738619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd4c5e-3317-492c-b20e-f0c19580c5a0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:49.758424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:49.850706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd4c5e-3317-492c-b20e-f0c19580c5a0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:49.885406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac91fcf-1df4-49ed-88db-e511aaf09e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:49.886036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91fcf-1df4-49ed-88db-e511aaf09e0e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:49.886318 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91fcf-1df4-49ed-88db-e511aaf09e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:49.886590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac91fcf-1df4-49ed-88db-e511aaf09e0e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:49.898836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac91fcf-1df4-49ed-88db-e511aaf09e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-fac91fcf-1df4-49ed-88db-e511aaf09e0e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:49.903513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:49.904493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:49.909128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 529/1062] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 13:19:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:49.912252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15bb7a-f117-4d5f-aaa8-4bc0a6d21e8c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:49.946799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:50.032500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15bb7a-f117-4d5f-aaa8-4bc0a6d21e8c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:50.070095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:50.070718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.071001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.071276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.071958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:50.072291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:50.073678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:50.082971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 534/1063] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 13:19:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:50.088676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac91fcf-1df4-49ed-88db-e511aaf09e0e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:50.170643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:50.198362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac91fcf-1df4-49ed-88db-e511aaf09e0e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:50.233457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9721061-7999-4fa3-9294-f2aa75cfedc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:50.234156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9721061-7999-4fa3-9294-f2aa75cfedc0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.234402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9721061-7999-4fa3-9294-f2aa75cfedc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.234661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9721061-7999-4fa3-9294-f2aa75cfedc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.245556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9721061-7999-4fa3-9294-f2aa75cfedc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:50.247001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9721061-7999-4fa3-9294-f2aa75cfedc0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a9721061-7999-4fa3-9294-f2aa75cfedc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:50.262324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:50.263308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:50.264268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:50.264268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:50.264268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:50.268274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 530/1064] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:50.269726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:50.383536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:50.396267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838808b-7302-4535-b59d-d2b51105f6af None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.400315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838808b-7302-4535-b59d-d2b51105f6af None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.400315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838808b-7302-4535-b59d-d2b51105f6af None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.400501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b067a8-87f4-4d1d-a738-1c2f9ea9b30c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:50.400590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 531/1065] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:50.403124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ba015-e03d-4a80-9f60-49a6e3e1fef3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.403414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ba015-e03d-4a80-9f60-49a6e3e1fef3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.403690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ba015-e03d-4a80-9f60-49a6e3e1fef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.422404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ef5ba015-e03d-4a80-9f60-49a6e3e1fef3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:50.423847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ef5ba015-e03d-4a80-9f60-49a6e3e1fef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:50.436088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:50.437790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.438247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.438519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.439340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:50.440041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:50.441444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:50.454626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 535/1066] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:50.460225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:50.500381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 532/1067] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:50.573282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa20548-9d80-4d38-a7a5-272ffdd9bcf0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:50.607270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:50.607954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.608170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.609002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.616833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:50.618276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:50.629952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:50.631233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:50.632348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:50.636118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 536/1068] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:50.637101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ba015-e03d-4a80-9f60-49a6e3e1fef3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:50.696849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:50.740394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ba015-e03d-4a80-9f60-49a6e3e1fef3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:50.770594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:50.777641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:50.778258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.778538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.778811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.779545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:50.779850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:50.781590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:50.786801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:50.802451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 533/1069] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:50.807998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:50.882539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d128bfc-b906-4b3d-b8d4-f4ec432201ae None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:50.916487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4d17d-e4e6-47da-bd0e-efabfc7aa87a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:50.919249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63947e0-dac3-4d95-ad23-49e9e73e9454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:50.919942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63947e0-dac3-4d95-ad23-49e9e73e9454 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.920324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63947e0-dac3-4d95-ad23-49e9e73e9454 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.920665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63947e0-dac3-4d95-ad23-49e9e73e9454 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.932310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63947e0-dac3-4d95-ad23-49e9e73e9454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:50.935620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e63947e0-dac3-4d95-ad23-49e9e73e9454 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:50.937287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:50.938561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:50.939829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:50.940561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 537/1070] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:50.945621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63947e0-dac3-4d95-ad23-49e9e73e9454 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:50.961655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:50.962379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:50.962659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:50.962935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:50.974251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:50.975754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:50.991323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:50.992600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:50.993834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:50.993834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:50.993834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:50.997381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 534/1071] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:50.998246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:51.028678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:51.060897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63947e0-dac3-4d95-ad23-49e9e73e9454 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.096661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe904f79-ab1a-4bbd-ae47-f1f150098fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.097266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe904f79-ab1a-4bbd-ae47-f1f150098fcf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.097547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe904f79-ab1a-4bbd-ae47-f1f150098fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.097824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe904f79-ab1a-4bbd-ae47-f1f150098fcf None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.098497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe904f79-ab1a-4bbd-ae47-f1f150098fcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:51.098803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe904f79-ab1a-4bbd-ae47-f1f150098fcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.100264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe904f79-ab1a-4bbd-ae47-f1f150098fcf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.103668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c306a68-5fd9-4bef-a3ce-b430776e5bf4 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.114672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 538/1072] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:51.114672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fccdb-4837-4cc3-883b-f1f19ae3ec60 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.114672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fccdb-4837-4cc3-883b-f1f19ae3ec60 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.114672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fccdb-4837-4cc3-883b-f1f19ae3ec60 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.115177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 539/1073] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:51.115741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fccdb-4837-4cc3-883b-f1f19ae3ec60 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:51.140490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc74c5-480b-414e-87fd-8b60801be405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.141085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc74c5-480b-414e-87fd-8b60801be405 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.141390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc74c5-480b-414e-87fd-8b60801be405 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.141663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc74c5-480b-414e-87fd-8b60801be405 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.142326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc74c5-480b-414e-87fd-8b60801be405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:51.142625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc74c5-480b-414e-87fd-8b60801be405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.143972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc74c5-480b-414e-87fd-8b60801be405 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.156161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 535/1074] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:51.157461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556054-a111-4282-aed5-e7ba68e4227b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.157822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556054-a111-4282-aed5-e7ba68e4227b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.158167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556054-a111-4282-aed5-e7ba68e4227b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.176737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f7556054-a111-4282-aed5-e7ba68e4227b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:51.177852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f7556054-a111-4282-aed5-e7ba68e4227b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:51.201097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:51.222637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fccdb-4837-4cc3-883b-f1f19ae3ec60 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.246642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 536/1075] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:51.252558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556054-a111-4282-aed5-e7ba68e4227b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:51.257731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325a3e7-8ddb-4917-9a55-f4e347632d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.258369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325a3e7-8ddb-4917-9a55-f4e347632d92 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.258651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325a3e7-8ddb-4917-9a55-f4e347632d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.258925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325a3e7-8ddb-4917-9a55-f4e347632d92 None admin] PATH_INFO: `/v3/users/ae7c53111fac47aaa0bb4d380f523101` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.260089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325a3e7-8ddb-4917-9a55-f4e347632d92 None admin] RBAC: Authorizing `identity:get_user(user_id=ae7c53111fac47aaa0bb4d380f523101)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.261801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325a3e7-8ddb-4917-9a55-f4e347632d92 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.263454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 540/1076] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:19:51 2022] GET /identity/v3/users/ae7c53111fac47aaa0bb4d380f523101 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:51.271680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325a3e7-8ddb-4917-9a55-f4e347632d92 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:51.274931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:51.373457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556054-a111-4282-aed5-e7ba68e4227b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.416724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325a3e7-8ddb-4917-9a55-f4e347632d92 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.430862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.431400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.431670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.431944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.440093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.441268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:51.453669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:51.454818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:51.455907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 537/1077] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:51.462513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:51.466563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6081f6-eddd-46bd-be9c-48dcf8813c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.467204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6081f6-eddd-46bd-be9c-48dcf8813c54 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.467486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6081f6-eddd-46bd-be9c-48dcf8813c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.468544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6081f6-eddd-46bd-be9c-48dcf8813c54 None admin] PATH_INFO: `/v3/projects/61e1e75e0707484f999804a7d9404c51` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.469637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6081f6-eddd-46bd-be9c-48dcf8813c54 None admin] RBAC: Authorizing `identity:get_project(project_id=61e1e75e0707484f999804a7d9404c51)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.471074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6081f6-eddd-46bd-be9c-48dcf8813c54 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.478069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 541/1078] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:19:51 2022] GET /identity/v3/projects/61e1e75e0707484f999804a7d9404c51 => generated 311 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:51.478912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6081f6-eddd-46bd-be9c-48dcf8813c54 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:51.485384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:51.569922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:51.580571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf83df-3b5b-4314-b85a-9d727e71958b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.604144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6081f6-eddd-46bd-be9c-48dcf8813c54 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.616568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.617168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.617446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.617716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.618406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:51.619021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.620376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.634600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 538/1079] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:51.635572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:51.653471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83560aac-395b-4624-a7c9-403f2bb70b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.654387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560aac-395b-4624-a7c9-403f2bb70b37 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.654675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560aac-395b-4624-a7c9-403f2bb70b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.655911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560aac-395b-4624-a7c9-403f2bb70b37 None admin] PATH_INFO: `/v3/projects/61e1e75e0707484f999804a7d9404c51/users/ae7c53111fac47aaa0bb4d380f523101/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.658990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83560aac-395b-4624-a7c9-403f2bb70b37 None admin] RBAC: Authorizing `identity:create_grant(project_id=61e1e75e0707484f999804a7d9404c51, user_id=ae7c53111fac47aaa0bb4d380f523101, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.660555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83560aac-395b-4624-a7c9-403f2bb70b37 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.723166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:51.737081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 542/1080] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:19:51 2022] PUT /identity/v3/projects/61e1e75e0707484f999804a7d9404c51/users/ae7c53111fac47aaa0bb4d380f523101/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:19:51.737901 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83560aac-395b-4624-a7c9-403f2bb70b37 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:51.756313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4c03e-778a-4693-91e4-756d17cc7dbc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.805692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9682982-96d8-419e-858b-5e41b5afebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.806692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682982-96d8-419e-858b-5e41b5afebef None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.807407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682982-96d8-419e-858b-5e41b5afebef None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.807889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9682982-96d8-419e-858b-5e41b5afebef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.912561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9682982-96d8-419e-858b-5e41b5afebef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.914020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9682982-96d8-419e-858b-5e41b5afebef None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.927515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83560aac-395b-4624-a7c9-403f2bb70b37 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:51.959624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:51.966038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873b72f-3d69-4d72-89f3-bcaad7d1b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:51.980428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873b72f-3d69-4d72-89f3-bcaad7d1b18b None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:51.980428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873b72f-3d69-4d72-89f3-bcaad7d1b18b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:51.980428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873b72f-3d69-4d72-89f3-bcaad7d1b18b None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/40561a306123461d92bc669be3962db3/roles/2a934a9192c84ef3a65ed7acfc442b2f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:51.980428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873b72f-3d69-4d72-89f3-bcaad7d1b18b None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=40561a306123461d92bc669be3962db3, role_id=2a934a9192c84ef3a65ed7acfc442b2f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:51.980428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873b72f-3d69-4d72-89f3-bcaad7d1b18b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:51.997545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:52.019642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 539/1081] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:19:52.025472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9682982-96d8-419e-858b-5e41b5afebef None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:52.066476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 543/1082] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:19:51 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/40561a306123461d92bc669be3962db3/roles/2a934a9192c84ef3a65ed7acfc442b2f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:19:52.165700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9682982-96d8-419e-858b-5e41b5afebef None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:52.205962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79d3a6-4e4d-4959-88c5-6fb52e9d5dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:52.206607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79d3a6-4e4d-4959-88c5-6fb52e9d5dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:52.206887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79d3a6-4e4d-4959-88c5-6fb52e9d5dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:52.207160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79d3a6-4e4d-4959-88c5-6fb52e9d5dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:52.221737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79d3a6-4e4d-4959-88c5-6fb52e9d5dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:52.223261 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79d3a6-4e4d-4959-88c5-6fb52e9d5dc6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3c79d3a6-4e4d-4959-88c5-6fb52e9d5dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:52.239980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:52.240994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:52.241898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:52.241898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:52.241898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:52.241898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 540/1083] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:52.247222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873b72f-3d69-4d72-89f3-bcaad7d1b18b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:52.354306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873b72f-3d69-4d72-89f3-bcaad7d1b18b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:52.389435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91d099-2d72-47c0-ba6f-7ddf85e9004f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:52.390703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91d099-2d72-47c0-ba6f-7ddf85e9004f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:52.390934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91d099-2d72-47c0-ba6f-7ddf85e9004f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:52.391262 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91d099-2d72-47c0-ba6f-7ddf85e9004f None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:52.391921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91d099-2d72-47c0-ba6f-7ddf85e9004f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:52.392250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91d099-2d72-47c0-ba6f-7ddf85e9004f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:52.393645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91d099-2d72-47c0-ba6f-7ddf85e9004f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:52.406775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 544/1084] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:53.681258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9f8bf-67b7-4faa-9d9f-c58478002db1 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:53.681815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9f8bf-67b7-4faa-9d9f-c58478002db1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:53.681815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9f8bf-67b7-4faa-9d9f-c58478002db1 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:53.683137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 541/1085] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:53.689386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cbfe5-ac07-4d39-bbb2-7c353c567877 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:53.689837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cbfe5-ac07-4d39-bbb2-7c353c567877 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:53.690090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cbfe5-ac07-4d39-bbb2-7c353c567877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:53.708038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-957cbfe5-ac07-4d39-bbb2-7c353c567877 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:53.709389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-957cbfe5-ac07-4d39-bbb2-7c353c567877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:53.757295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:53.777375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 545/1086] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:54.058303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9f8bf-67b7-4faa-9d9f-c58478002db1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.068872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a00b9c-9174-4d84-8cae-ff4c1ddfad48 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.069169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a00b9c-9174-4d84-8cae-ff4c1ddfad48 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.069451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a00b9c-9174-4d84-8cae-ff4c1ddfad48 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.072367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 546/1087] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:19:54.076405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958598e2-5357-4fcb-9069-5a96d33e1718 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.076693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958598e2-5357-4fcb-9069-5a96d33e1718 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.076982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958598e2-5357-4fcb-9069-5a96d33e1718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.090857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:54.095534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-958598e2-5357-4fcb-9069-5a96d33e1718 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:54.096778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-958598e2-5357-4fcb-9069-5a96d33e1718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:54.164196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9f8bf-67b7-4faa-9d9f-c58478002db1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:54.165967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 547/1088] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:19:54.199988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb9d6-3fb1-4652-98c6-12194b45670a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:54.200649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb9d6-3fb1-4652-98c6-12194b45670a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.201024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb9d6-3fb1-4652-98c6-12194b45670a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.201301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb9d6-3fb1-4652-98c6-12194b45670a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.212121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8eb9d6-3fb1-4652-98c6-12194b45670a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-ed8eb9d6-3fb1-4652-98c6-12194b45670a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:54.216766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:54.218280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:19:54.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:54.220481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 542/1089] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:54.223814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958598e2-5357-4fcb-9069-5a96d33e1718 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.323061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958598e2-5357-4fcb-9069-5a96d33e1718 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:54.333682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:54.357034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:54.357624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.357904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.358862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.359698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:54.360005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:54.361430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:54.369427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 548/1090] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:54.376210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb9d6-3fb1-4652-98c6-12194b45670a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.439504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.468438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:54.478564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb9d6-3fb1-4652-98c6-12194b45670a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:54.513308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e39302-5f8f-4e43-8c6b-3961a7ef15c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:54.513937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e39302-5f8f-4e43-8c6b-3961a7ef15c7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.514219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e39302-5f8f-4e43-8c6b-3961a7ef15c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.514523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e39302-5f8f-4e43-8c6b-3961a7ef15c7 None admin] PATH_INFO: `/v3/users/ae7c53111fac47aaa0bb4d380f523101` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.515709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e39302-5f8f-4e43-8c6b-3961a7ef15c7 None admin] RBAC: Authorizing `identity:get_user(user_id=ae7c53111fac47aaa0bb4d380f523101)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:54.517175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e39302-5f8f-4e43-8c6b-3961a7ef15c7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:54.518885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 543/1091] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/users/ae7c53111fac47aaa0bb4d380f523101 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:54.524859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e39302-5f8f-4e43-8c6b-3961a7ef15c7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.539828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4776bfe-993c-47cf-b6c4-3aa7956d02f8 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:54.575438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b273b2c-3968-4fdd-ac47-a6b31b8899df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:54.575981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b273b2c-3968-4fdd-ac47-a6b31b8899df None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.576292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b273b2c-3968-4fdd-ac47-a6b31b8899df None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.576566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b273b2c-3968-4fdd-ac47-a6b31b8899df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.587605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b273b2c-3968-4fdd-ac47-a6b31b8899df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7b273b2c-3968-4fdd-ac47-a6b31b8899df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:19:54.592325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:54.594005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 13:19:54.594890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:54.599425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 549/1092] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:54.600291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b273b2c-3968-4fdd-ac47-a6b31b8899df None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.631176 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e39302-5f8f-4e43-8c6b-3961a7ef15c7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:54.635136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:54.666577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7ce4c-308c-4f5e-bfc8-00c14483d104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:54.667185 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ce4c-308c-4f5e-bfc8-00c14483d104 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.667470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ce4c-308c-4f5e-bfc8-00c14483d104 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.668007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ce4c-308c-4f5e-bfc8-00c14483d104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.696124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:54.707445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b273b2c-3968-4fdd-ac47-a6b31b8899df None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:54.740501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7ce4c-308c-4f5e-bfc8-00c14483d104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:54.741825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7ce4c-308c-4f5e-bfc8-00c14483d104 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:54.742422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:54.743484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.743484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.743768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.744277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:54.744587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:54.745949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:54.756529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 550/1093] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:54.762154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.847487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 544/1094] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 13:19:54.853452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7ce4c-308c-4f5e-bfc8-00c14483d104 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.857051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:54.864042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83319be6-d275-4261-a731-edef8c448ac3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:54.899919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:54.900580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.900866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.901141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.911174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:54.911566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:54.913112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:54.928882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:54.931355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:54.932312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:54.932312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:54.932312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:54.932312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 551/1095] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:54.936008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:54.959421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7ce4c-308c-4f5e-bfc8-00c14483d104 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:54.994613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8e523-3cbc-4cf7-9275-aa7dbe502e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:54.995237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8e523-3cbc-4cf7-9275-aa7dbe502e85 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:54.995519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8e523-3cbc-4cf7-9275-aa7dbe502e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:54.995794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8e523-3cbc-4cf7-9275-aa7dbe502e85 None admin] PATH_INFO: `/v3/projects/61e1e75e0707484f999804a7d9404c51` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:54.996946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8e523-3cbc-4cf7-9275-aa7dbe502e85 None admin] RBAC: Authorizing `identity:get_project(project_id=61e1e75e0707484f999804a7d9404c51)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:54.998390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8e523-3cbc-4cf7-9275-aa7dbe502e85 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:55.000807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 545/1096] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/projects/61e1e75e0707484f999804a7d9404c51 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:55.006888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8e523-3cbc-4cf7-9275-aa7dbe502e85 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:55.038612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80fe7f-6882-4b38-a2d0-0811c8209499 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:55.073679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:55.074275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:55.074550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:55.074820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:55.075450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:55.075747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:55.077321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:55.089187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 552/1097] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:55.095025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:55.103431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:55.107996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8e523-3cbc-4cf7-9275-aa7dbe502e85 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:55.135015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:55.143792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:55.144416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:55.144704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:55.144980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:55.145708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=61e1e75e0707484f999804a7d9404c51, user.id=ae7c53111fac47aaa0bb4d380f523101) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:55.146015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:55.147446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:55.153276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 546/1098] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:55 2022] GET /identity/v3/role_assignments?user.id=ae7c53111fac47aaa0bb4d380f523101&scope.project.id=61e1e75e0707484f999804a7d9404c51&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:55.199762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87e562-ba55-46ad-a530-7a135dcffef7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:55.238748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:55.239385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:55.239723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:55.239930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:55.247780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:55.249901 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:19:55.262377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:55.263433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:19:55.264463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 553/1099] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 13:19:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:55.269769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:55.373488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602fdc-8969-40be-bffd-b17287c561cd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:55.407464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:55.408110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:55.408420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:55.408694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:55.409420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:55.409727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:55.411143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:55.429778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 547/1100] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 13:19:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:55.435597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:55.503263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:55.540431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6774f41-6ab2-483c-bc64-b902cd1907ef None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:55.574744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d870706-bc8e-4b84-9c0f-fd41a5675cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:55.575347 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d870706-bc8e-4b84-9c0f-fd41a5675cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:55.575629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d870706-bc8e-4b84-9c0f-fd41a5675cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:55.575910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d870706-bc8e-4b84-9c0f-fd41a5675cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:55.648006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d870706-bc8e-4b84-9c0f-fd41a5675cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:55.649423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d870706-bc8e-4b84-9c0f-fd41a5675cf1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:55.715204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:55.752337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 554/1101] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:19:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:19:55.756692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:55.806596 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:55.856836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44eca9-b48d-4129-b8d6-388897d0a1ed None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:55.892213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:55.892827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:55.893110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:55.893382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:55.903236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:55.904700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:19:55.919480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:55.921219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:19:55.922878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:19:55.922878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:19:55.922878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:55.924346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 548/1102] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:19:55.927188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d870706-bc8e-4b84-9c0f-fd41a5675cf1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:56.025805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d870706-bc8e-4b84-9c0f-fd41a5675cf1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:56.060480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:56.061084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:56.062006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:56.062283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:56.062916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:56.063214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:56.064648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:56.075825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 555/1103] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:56.081515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:56.181344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:56.184203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30f2e0-1d94-4ba6-9d30-2f02e196f49c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:56.219379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:56.220071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:56.220342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:56.220615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:56.227570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:56.229035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:19:56.242094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:19:56.248160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:19:56.249370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 549/1104] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:19:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:19:56.249855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:56.278536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:56.352023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352e56a-c9a8-451a-9b05-a3f3c5f4abd4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:56.385285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71f7e0-071b-4f73-9db9-3271384c678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:56.385884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71f7e0-071b-4f73-9db9-3271384c678f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:56.386169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71f7e0-071b-4f73-9db9-3271384c678f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:56.386442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71f7e0-071b-4f73-9db9-3271384c678f None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:56.387134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71f7e0-071b-4f73-9db9-3271384c678f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:56.387446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71f7e0-071b-4f73-9db9-3271384c678f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:56.388919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71f7e0-071b-4f73-9db9-3271384c678f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:56.401195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 556/1105] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:19:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:56.407967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:56.508815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8f1a3-afd0-4f4e-b018-3b0c7fbbee13 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:56.548892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e08aa-c6ad-4281-a759-4784bb160e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:56.549619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e08aa-c6ad-4281-a759-4784bb160e75 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:56.549823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e08aa-c6ad-4281-a759-4784bb160e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:56.550064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e08aa-c6ad-4281-a759-4784bb160e75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:56.550739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e08aa-c6ad-4281-a759-4784bb160e75 None admin] RBAC: Adding query filter params (role.id=2a934a9192c84ef3a65ed7acfc442b2f, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=40561a306123461d92bc669be3962db3) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:19:56.551044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e08aa-c6ad-4281-a759-4784bb160e75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:56.552556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e08aa-c6ad-4281-a759-4784bb160e75 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:56.558584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 550/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:19:56 2022] GET /identity/v3/role_assignments?user.id=40561a306123461d92bc669be3962db3&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=2a934a9192c84ef3a65ed7acfc442b2f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:19:57.163956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83531e94-201c-4bfb-b1ff-5a176984e510 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:57.164517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83531e94-201c-4bfb-b1ff-5a176984e510 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:57.164517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83531e94-201c-4bfb-b1ff-5a176984e510 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:57.165844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 557/1107] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:19:57.171821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e41615-bd88-484e-b276-0bb724e0bf40 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:57.172291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e41615-bd88-484e-b276-0bb724e0bf40 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:57.172426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e41615-bd88-484e-b276-0bb724e0bf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:57.191989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b5e41615-bd88-484e-b276-0bb724e0bf40 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:19:57.193231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b5e41615-bd88-484e-b276-0bb724e0bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:19:57.213414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:57.269676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 551/1108] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:19:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:19:57.550678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83531e94-201c-4bfb-b1ff-5a176984e510 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:57.648557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:19:57.665169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83531e94-201c-4bfb-b1ff-5a176984e510 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:57.700550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e53b5c-2bf3-4287-918f-f1646c19d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:57.701171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e53b5c-2bf3-4287-918f-f1646c19d816 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:57.701446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e53b5c-2bf3-4287-918f-f1646c19d816 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:57.701714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e53b5c-2bf3-4287-918f-f1646c19d816 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:57.703737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e53b5c-2bf3-4287-918f-f1646c19d816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:57.705184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e53b5c-2bf3-4287-918f-f1646c19d816 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:57.706752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 558/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:19:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:19:57.712726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e41615-bd88-484e-b276-0bb724e0bf40 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:19:57.820852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e41615-bd88-484e-b276-0bb724e0bf40 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:19:57.856739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad312e72-2bea-47f9-89a6-db8562c5c44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:19:57.857342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312e72-2bea-47f9-89a6-db8562c5c44b None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:19:57.857626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312e72-2bea-47f9-89a6-db8562c5c44b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:19:57.857901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad312e72-2bea-47f9-89a6-db8562c5c44b None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:19:57.859149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad312e72-2bea-47f9-89a6-db8562c5c44b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:19:57.860612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad312e72-2bea-47f9-89a6-db8562c5c44b None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:19:57.962564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 552/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:19:57 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:20:00.003372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c3371-0109-4f75-88b4-bb26ef0e4883 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:00.003896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c3371-0109-4f75-88b4-bb26ef0e4883 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:00.003896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c3371-0109-4f75-88b4-bb26ef0e4883 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:00.005419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 559/1111] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:00.010782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d026cd-0c8b-4c66-a4fa-73e7fc874da3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:00.011180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d026cd-0c8b-4c66-a4fa-73e7fc874da3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:00.011401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d026cd-0c8b-4c66-a4fa-73e7fc874da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:00.030858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-00d026cd-0c8b-4c66-a4fa-73e7fc874da3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:00.032201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-00d026cd-0c8b-4c66-a4fa-73e7fc874da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:00.034933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:00.100107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 553/1112] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:00.400232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c3371-0109-4f75-88b4-bb26ef0e4883 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:00.501813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c3371-0109-4f75-88b4-bb26ef0e4883 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:00.538790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f53cc7-5a42-4b88-b830-a45788ed647a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:00.539437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f53cc7-5a42-4b88-b830-a45788ed647a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:00.539722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f53cc7-5a42-4b88-b830-a45788ed647a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:00.539996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f53cc7-5a42-4b88-b830-a45788ed647a None admin] PATH_INFO: `/v3/roles/47878794de344232a226a95ce5771504` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:00.541535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f53cc7-5a42-4b88-b830-a45788ed647a None admin] RBAC: Authorizing `identity:get_role(role_id=47878794de344232a226a95ce5771504)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:00.542924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f53cc7-5a42-4b88-b830-a45788ed647a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:00.544464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 560/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:00 2022] GET /identity/v3/roles/47878794de344232a226a95ce5771504 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:00.551637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d026cd-0c8b-4c66-a4fa-73e7fc874da3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:00.718503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d026cd-0c8b-4c66-a4fa-73e7fc874da3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:00.752737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0df464-b271-46f4-9c49-e145797ec141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:00.753401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df464-b271-46f4-9c49-e145797ec141 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:00.753682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df464-b271-46f4-9c49-e145797ec141 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:00.753963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df464-b271-46f4-9c49-e145797ec141 None admin] PATH_INFO: `/v3/users/9b115ff3b5284db48a73b01a3839b69f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:00.764613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:00.767756 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0df464-b271-46f4-9c49-e145797ec141 None admin] RBAC: Authorizing `identity:get_user(user_id=9b115ff3b5284db48a73b01a3839b69f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:00.769289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0df464-b271-46f4-9c49-e145797ec141 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:00.770897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 554/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:00 2022] GET /identity/v3/users/9b115ff3b5284db48a73b01a3839b69f => generated 286 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:00.776949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f53cc7-5a42-4b88-b830-a45788ed647a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:00.802019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:00.878673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f53cc7-5a42-4b88-b830-a45788ed647a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:00.912249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e33a12-1e1e-4434-83a2-caf7288e1eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:00.912857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e33a12-1e1e-4434-83a2-caf7288e1eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:00.913137 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e33a12-1e1e-4434-83a2-caf7288e1eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:00.913406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e33a12-1e1e-4434-83a2-caf7288e1eb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:00.979632 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e33a12-1e1e-4434-83a2-caf7288e1eb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:00.981023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e33a12-1e1e-4434-83a2-caf7288e1eb7 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:01.010735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:01.085631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 561/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:20:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:20:01.091743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0df464-b271-46f4-9c49-e145797ec141 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:01.190869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0df464-b271-46f4-9c49-e145797ec141 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:01.223694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6757eb35-7e7b-4628-95ab-dbcf2df4215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:01.224341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757eb35-7e7b-4628-95ab-dbcf2df4215c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:01.224622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757eb35-7e7b-4628-95ab-dbcf2df4215c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:01.224895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6757eb35-7e7b-4628-95ab-dbcf2df4215c None admin] PATH_INFO: `/v3/projects/61e1e75e0707484f999804a7d9404c51` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:01.226011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757eb35-7e7b-4628-95ab-dbcf2df4215c None admin] RBAC: Authorizing `identity:get_project(project_id=61e1e75e0707484f999804a7d9404c51)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:01.227425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6757eb35-7e7b-4628-95ab-dbcf2df4215c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:01.229066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 555/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:01 2022] GET /identity/v3/projects/61e1e75e0707484f999804a7d9404c51 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:01.235947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e33a12-1e1e-4434-83a2-caf7288e1eb7 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:01.332833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e33a12-1e1e-4434-83a2-caf7288e1eb7 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:01.354689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:01.367031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a8cbc-5085-4ce8-82d2-4923e6fc6609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:01.367639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a8cbc-5085-4ce8-82d2-4923e6fc6609 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:01.367914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a8cbc-5085-4ce8-82d2-4923e6fc6609 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:01.368203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a8cbc-5085-4ce8-82d2-4923e6fc6609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:01.368940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a8cbc-5085-4ce8-82d2-4923e6fc6609 None admin] RBAC: Adding query filter params (role.id=47878794de344232a226a95ce5771504, scope.project.id=61e1e75e0707484f999804a7d9404c51, user.id=9b115ff3b5284db48a73b01a3839b69f) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:01.369299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a8cbc-5085-4ce8-82d2-4923e6fc6609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:01.370722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a8cbc-5085-4ce8-82d2-4923e6fc6609 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:01.376114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 562/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:20:01 2022] GET /identity/v3/role_assignments?user.id=9b115ff3b5284db48a73b01a3839b69f&scope.project.id=61e1e75e0707484f999804a7d9404c51&role.id=47878794de344232a226a95ce5771504 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:03.269656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcffb9-455a-4b8a-ba12-a30f380ce9bd None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:03.270196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcffb9-455a-4b8a-ba12-a30f380ce9bd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:03.270196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcffb9-455a-4b8a-ba12-a30f380ce9bd None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:03.271530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 556/1118] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:03.276619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faab4f6-b264-4456-8535-95b9288e6021 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:03.277065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faab4f6-b264-4456-8535-95b9288e6021 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:03.277158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faab4f6-b264-4456-8535-95b9288e6021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:03.296026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7faab4f6-b264-4456-8535-95b9288e6021 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:03.297274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7faab4f6-b264-4456-8535-95b9288e6021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:03.366288 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 563/1119] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:20:03.634142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcffb9-455a-4b8a-ba12-a30f380ce9bd None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:03.700887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:03.736794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcffb9-455a-4b8a-ba12-a30f380ce9bd None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:03.774600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0277820-8666-4546-b2c3-476b9d44ecbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:03.775222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277820-8666-4546-b2c3-476b9d44ecbf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:03.775506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277820-8666-4546-b2c3-476b9d44ecbf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:03.775784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277820-8666-4546-b2c3-476b9d44ecbf None admin] PATH_INFO: `/v3/roles/47878794de344232a226a95ce5771504` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:03.777247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0277820-8666-4546-b2c3-476b9d44ecbf None admin] RBAC: Authorizing `identity:get_role(role_id=47878794de344232a226a95ce5771504)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:03.778610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0277820-8666-4546-b2c3-476b9d44ecbf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:03.780021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 557/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:03 2022] GET /identity/v3/roles/47878794de344232a226a95ce5771504 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:20:03.785907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faab4f6-b264-4456-8535-95b9288e6021 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:03.886154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faab4f6-b264-4456-8535-95b9288e6021 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:03.902821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:03.919922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334e14-f786-4b98-9560-571a6409e8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:03.921096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334e14-f786-4b98-9560-571a6409e8de None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:03.921378 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334e14-f786-4b98-9560-571a6409e8de None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:03.921663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334e14-f786-4b98-9560-571a6409e8de None admin] PATH_INFO: `/v3/users/9b115ff3b5284db48a73b01a3839b69f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:03.922928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65334e14-f786-4b98-9560-571a6409e8de None admin] RBAC: Authorizing `identity:get_user(user_id=9b115ff3b5284db48a73b01a3839b69f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:03.924653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65334e14-f786-4b98-9560-571a6409e8de None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:03.926216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 564/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:03 2022] GET /identity/v3/users/9b115ff3b5284db48a73b01a3839b69f => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:03.932075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0277820-8666-4546-b2c3-476b9d44ecbf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:04.030348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0277820-8666-4546-b2c3-476b9d44ecbf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:04.064471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60185013-2db8-45eb-b819-a758c2248915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:04.065088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60185013-2db8-45eb-b819-a758c2248915 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:04.065369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60185013-2db8-45eb-b819-a758c2248915 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:04.065643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60185013-2db8-45eb-b819-a758c2248915 None admin] PATH_INFO: `/v3/projects/61e1e75e0707484f999804a7d9404c51` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:04.066770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60185013-2db8-45eb-b819-a758c2248915 None admin] RBAC: Authorizing `identity:get_project(project_id=61e1e75e0707484f999804a7d9404c51)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:04.068216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60185013-2db8-45eb-b819-a758c2248915 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:04.069838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 558/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:03 2022] GET /identity/v3/projects/61e1e75e0707484f999804a7d9404c51 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:04.075825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334e14-f786-4b98-9560-571a6409e8de None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:04.179591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334e14-f786-4b98-9560-571a6409e8de None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:04.213533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07174c53-1a30-4db3-8314-4285aceb2b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:04.214187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07174c53-1a30-4db3-8314-4285aceb2b58 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:04.214462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07174c53-1a30-4db3-8314-4285aceb2b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:04.214735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07174c53-1a30-4db3-8314-4285aceb2b58 None admin] PATH_INFO: `/v3/projects/61e1e75e0707484f999804a7d9404c51/users/9b115ff3b5284db48a73b01a3839b69f/roles/47878794de344232a226a95ce5771504` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:04.216890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07174c53-1a30-4db3-8314-4285aceb2b58 None admin] RBAC: Authorizing `identity:create_grant(project_id=61e1e75e0707484f999804a7d9404c51, user_id=9b115ff3b5284db48a73b01a3839b69f, role_id=47878794de344232a226a95ce5771504)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:04.218432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07174c53-1a30-4db3-8314-4285aceb2b58 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:04.301824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 565/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:20:04 2022] PUT /identity/v3/projects/61e1e75e0707484f999804a7d9404c51/users/9b115ff3b5284db48a73b01a3839b69f/roles/47878794de344232a226a95ce5771504 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:06.205830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93e8f7-1173-4ea3-9aaf-5867e7ea03bc None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:06.206365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93e8f7-1173-4ea3-9aaf-5867e7ea03bc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:06.206365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93e8f7-1173-4ea3-9aaf-5867e7ea03bc None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:06.207723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 559/1124] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:06.213513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379df63-da64-4e55-a3c4-d9b467b13833 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:06.213983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379df63-da64-4e55-a3c4-d9b467b13833 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:06.214077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379df63-da64-4e55-a3c4-d9b467b13833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:06.232415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d379df63-da64-4e55-a3c4-d9b467b13833 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:06.234433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d379df63-da64-4e55-a3c4-d9b467b13833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:06.274455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:06.301455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 566/1125] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:06.581241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93e8f7-1173-4ea3-9aaf-5867e7ea03bc None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:06.584119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:06.682106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93e8f7-1173-4ea3-9aaf-5867e7ea03bc None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:06.715996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889638-7d92-4378-9132-9a011872637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:06.716768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889638-7d92-4378-9132-9a011872637c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:06.717730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889638-7d92-4378-9132-9a011872637c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:06.718266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889638-7d92-4378-9132-9a011872637c None admin] PATH_INFO: `/v3/roles/47878794de344232a226a95ce5771504` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:06.719812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53889638-7d92-4378-9132-9a011872637c None admin] RBAC: Authorizing `identity:get_role(role_id=47878794de344232a226a95ce5771504)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:06.721323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53889638-7d92-4378-9132-9a011872637c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:06.722636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 560/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:06 2022] GET /identity/v3/roles/47878794de344232a226a95ce5771504 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:06.728505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379df63-da64-4e55-a3c4-d9b467b13833 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:06.826919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379df63-da64-4e55-a3c4-d9b467b13833 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:06.862187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3071eb-5ce0-4137-95d5-55ced8c0599b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:06.862737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3071eb-5ce0-4137-95d5-55ced8c0599b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:06.863016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3071eb-5ce0-4137-95d5-55ced8c0599b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:06.863287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3071eb-5ce0-4137-95d5-55ced8c0599b None admin] PATH_INFO: `/v3/users/9b115ff3b5284db48a73b01a3839b69f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:06.864458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3071eb-5ce0-4137-95d5-55ced8c0599b None admin] RBAC: Authorizing `identity:get_user(user_id=9b115ff3b5284db48a73b01a3839b69f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:06.865897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3071eb-5ce0-4137-95d5-55ced8c0599b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:06.867465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 567/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:06 2022] GET /identity/v3/users/9b115ff3b5284db48a73b01a3839b69f => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:06.873645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889638-7d92-4378-9132-9a011872637c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:06.949255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:06.980353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889638-7d92-4378-9132-9a011872637c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:07.014513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15987c-170a-41ff-88ae-5befd659519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:07.015254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15987c-170a-41ff-88ae-5befd659519f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:07.015536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15987c-170a-41ff-88ae-5befd659519f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:07.015810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15987c-170a-41ff-88ae-5befd659519f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:07.082257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15987c-170a-41ff-88ae-5befd659519f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:07.083785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15987c-170a-41ff-88ae-5befd659519f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:07.149216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:07.180206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 561/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:20:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:20:07.186767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3071eb-5ce0-4137-95d5-55ced8c0599b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:07.231641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:07.295823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3071eb-5ce0-4137-95d5-55ced8c0599b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:07.330935 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c272da-9130-49d7-b729-8a289493ed0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:07.331560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c272da-9130-49d7-b729-8a289493ed0f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:07.331841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c272da-9130-49d7-b729-8a289493ed0f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:07.332115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c272da-9130-49d7-b729-8a289493ed0f None admin] PATH_INFO: `/v3/projects/61e1e75e0707484f999804a7d9404c51` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:07.333260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c272da-9130-49d7-b729-8a289493ed0f None admin] RBAC: Authorizing `identity:get_project(project_id=61e1e75e0707484f999804a7d9404c51)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:07.334691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c272da-9130-49d7-b729-8a289493ed0f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:07.336340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 568/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:07 2022] GET /identity/v3/projects/61e1e75e0707484f999804a7d9404c51 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:07.343707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15987c-170a-41ff-88ae-5befd659519f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:07.444624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15987c-170a-41ff-88ae-5befd659519f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:07.479095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d22274-3bc1-4a93-b4cd-f85cecbec24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:07.479685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22274-3bc1-4a93-b4cd-f85cecbec24a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:07.479966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22274-3bc1-4a93-b4cd-f85cecbec24a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:07.480268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d22274-3bc1-4a93-b4cd-f85cecbec24a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:07.480989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d22274-3bc1-4a93-b4cd-f85cecbec24a None admin] RBAC: Adding query filter params (role.id=47878794de344232a226a95ce5771504, scope.project.id=61e1e75e0707484f999804a7d9404c51, user.id=9b115ff3b5284db48a73b01a3839b69f) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:07.481297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d22274-3bc1-4a93-b4cd-f85cecbec24a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:07.483918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d22274-3bc1-4a93-b4cd-f85cecbec24a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:07.489593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 562/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:20:07 2022] GET /identity/v3/role_assignments?user.id=9b115ff3b5284db48a73b01a3839b69f&scope.project.id=61e1e75e0707484f999804a7d9404c51&role.id=47878794de344232a226a95ce5771504 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:09.512620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98497255-5221-4dfe-be32-85a1efb72885 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:09.513149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98497255-5221-4dfe-be32-85a1efb72885 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:09.513149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98497255-5221-4dfe-be32-85a1efb72885 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:09.514828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 569/1131] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:09.520409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5c74-cf0d-46c3-a3c7-4b7c8782f715 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:09.520826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5c74-cf0d-46c3-a3c7-4b7c8782f715 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:09.520973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a5c74-cf0d-46c3-a3c7-4b7c8782f715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:09.544699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2c8a5c74-cf0d-46c3-a3c7-4b7c8782f715 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:09.546538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2c8a5c74-cf0d-46c3-a3c7-4b7c8782f715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:09.574021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:09.614768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 563/1132] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:09.895014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98497255-5221-4dfe-be32-85a1efb72885 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:09.997717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98497255-5221-4dfe-be32-85a1efb72885 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:10.008468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:10.032236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53703f-2a56-43c9-b94e-eefd69e38b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:10.032858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53703f-2a56-43c9-b94e-eefd69e38b72 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:10.033139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53703f-2a56-43c9-b94e-eefd69e38b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:10.033411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53703f-2a56-43c9-b94e-eefd69e38b72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:10.034536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53703f-2a56-43c9-b94e-eefd69e38b72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:10.035962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53703f-2a56-43c9-b94e-eefd69e38b72 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:10.037926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 570/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:20:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:10.043790 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a5c74-cf0d-46c3-a3c7-4b7c8782f715 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:10.142861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a5c74-cf0d-46c3-a3c7-4b7c8782f715 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:10.176807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae927771-bca9-4ff7-865b-81b3bb9a6ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:10.177330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927771-bca9-4ff7-865b-81b3bb9a6ef4 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:10.177610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927771-bca9-4ff7-865b-81b3bb9a6ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:10.177885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae927771-bca9-4ff7-865b-81b3bb9a6ef4 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:10.178536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae927771-bca9-4ff7-865b-81b3bb9a6ef4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:10.179990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae927771-bca9-4ff7-865b-81b3bb9a6ef4 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:10.291063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 564/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:20:10 2022] POST /identity/v3/projects => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:12.212830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c745-feed-4d6a-90b1-528fbbd10820 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:12.213380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c745-feed-4d6a-90b1-528fbbd10820 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:12.213380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c745-feed-4d6a-90b1-528fbbd10820 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:12.214693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 571/1135] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:12.219953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2539-5c53-48bd-9ba9-720a6395dc94 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:12.220397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2539-5c53-48bd-9ba9-720a6395dc94 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:12.220552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2539-5c53-48bd-9ba9-720a6395dc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:12.239727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7cce2539-5c53-48bd-9ba9-720a6395dc94 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:12.240957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7cce2539-5c53-48bd-9ba9-720a6395dc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:12.252114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:12.309026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 565/1136] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:12.577333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c745-feed-4d6a-90b1-528fbbd10820 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:12.678091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c745-feed-4d6a-90b1-528fbbd10820 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:12.710805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d40fd6-c6a0-4085-9b9e-e2a203466256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:12.711416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d40fd6-c6a0-4085-9b9e-e2a203466256 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:12.711692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d40fd6-c6a0-4085-9b9e-e2a203466256 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:12.711958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d40fd6-c6a0-4085-9b9e-e2a203466256 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:12.713084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d40fd6-c6a0-4085-9b9e-e2a203466256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:12.714509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d40fd6-c6a0-4085-9b9e-e2a203466256 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:12.716054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 572/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:20:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:20:12.722700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce2539-5c53-48bd-9ba9-720a6395dc94 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:12.824936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce2539-5c53-48bd-9ba9-720a6395dc94 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:12.859958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3121d7-ebe8-49a2-91a3-e4b46a26c5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:12.860466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3121d7-ebe8-49a2-91a3-e4b46a26c5c6 None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:12.860749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3121d7-ebe8-49a2-91a3-e4b46a26c5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:12.861024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3121d7-ebe8-49a2-91a3-e4b46a26c5c6 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:12.861668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3121d7-ebe8-49a2-91a3-e4b46a26c5c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:12.863087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3121d7-ebe8-49a2-91a3-e4b46a26c5c6 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:12.963618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 566/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:20:12 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:14.915050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10443351-3883-45b7-854e-4417f58b924f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:14.915591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10443351-3883-45b7-854e-4417f58b924f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:14.915591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10443351-3883-45b7-854e-4417f58b924f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:14.916985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 573/1139] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:14.922546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f2bc9-67d2-45b8-9caa-c596b7df9c82 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:14.923002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f2bc9-67d2-45b8-9caa-c596b7df9c82 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:14.925733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f2bc9-67d2-45b8-9caa-c596b7df9c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:14.929747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:14.942833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2e5f2bc9-67d2-45b8-9caa-c596b7df9c82 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:14.944033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2e5f2bc9-67d2-45b8-9caa-c596b7df9c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:15.012867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 567/1140] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:15.292789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10443351-3883-45b7-854e-4417f58b924f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:15.341765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:15.397412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10443351-3883-45b7-854e-4417f58b924f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:15.432392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb436c-9f90-442d-9643-02d0029d9b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:15.433019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb436c-9f90-442d-9643-02d0029d9b4a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:15.433301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb436c-9f90-442d-9643-02d0029d9b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:15.433573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb436c-9f90-442d-9643-02d0029d9b4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:15.443883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb436c-9f90-442d-9643-02d0029d9b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-e1bb436c-9f90-442d-9643-02d0029d9b4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:20:15.448545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:20:15.449658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:20:15.450700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 574/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:20:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:20:15.456488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f2bc9-67d2-45b8-9caa-c596b7df9c82 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:15.554644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f2bc9-67d2-45b8-9caa-c596b7df9c82 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:15.585646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:15.590446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:15.591047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:15.591328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:15.591600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:15.592289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:15.592594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:15.594239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:15.607015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 568/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:20:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:15.615113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb436c-9f90-442d-9643-02d0029d9b4a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:15.724434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb436c-9f90-442d-9643-02d0029d9b4a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:15.731841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:15.759063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6e11e-0b98-484d-8d02-081ebbd72f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:15.759690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6e11e-0b98-484d-8d02-081ebbd72f37 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:15.759974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6e11e-0b98-484d-8d02-081ebbd72f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:15.760294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6e11e-0b98-484d-8d02-081ebbd72f37 None admin] PATH_INFO: `/v3/users/65e74fa4941a484e8bcfb4c386eb77f9` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:15.774858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6e11e-0b98-484d-8d02-081ebbd72f37 None admin] RBAC: Authorizing `identity:get_user(user_id=65e74fa4941a484e8bcfb4c386eb77f9)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:15.776325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6e11e-0b98-484d-8d02-081ebbd72f37 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:15.777939 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 575/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:15 2022] GET /identity/v3/users/65e74fa4941a484e8bcfb4c386eb77f9 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:15.784964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:15.886647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481374f5-b92e-4a25-84f0-4e82c20e78c2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:15.920323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a55e4-f119-415c-b09e-b9f993f04783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:15.920935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a55e4-f119-415c-b09e-b9f993f04783 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:15.921241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a55e4-f119-415c-b09e-b9f993f04783 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:15.921809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a55e4-f119-415c-b09e-b9f993f04783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:15.981437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:15.989960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a55e4-f119-415c-b09e-b9f993f04783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:15.991421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a55e4-f119-415c-b09e-b9f993f04783 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:16.089834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 569/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:20:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:20:16.096474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6e11e-0b98-484d-8d02-081ebbd72f37 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:16.194904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6e11e-0b98-484d-8d02-081ebbd72f37 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:16.230491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21de966-25d1-48f8-b025-7b9743b1346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:16.231107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21de966-25d1-48f8-b025-7b9743b1346b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:16.231497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21de966-25d1-48f8-b025-7b9743b1346b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:16.231783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21de966-25d1-48f8-b025-7b9743b1346b None admin] PATH_INFO: `/v3/projects/cdbd5dff18fd46ab808a6248f179f3c9` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:16.232934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21de966-25d1-48f8-b025-7b9743b1346b None admin] RBAC: Authorizing `identity:get_project(project_id=cdbd5dff18fd46ab808a6248f179f3c9)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:16.234423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21de966-25d1-48f8-b025-7b9743b1346b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:16.236049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 576/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:16 2022] GET /identity/v3/projects/cdbd5dff18fd46ab808a6248f179f3c9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:16.242290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a55e4-f119-415c-b09e-b9f993f04783 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:16.337365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:16.344429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a55e4-f119-415c-b09e-b9f993f04783 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:16.378143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034bdd0-9387-4e71-947a-e8acb17b6720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:16.378738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bdd0-9387-4e71-947a-e8acb17b6720 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:16.379019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bdd0-9387-4e71-947a-e8acb17b6720 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:16.379289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bdd0-9387-4e71-947a-e8acb17b6720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:16.380008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034bdd0-9387-4e71-947a-e8acb17b6720 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=cdbd5dff18fd46ab808a6248f179f3c9, user.id=65e74fa4941a484e8bcfb4c386eb77f9) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:16.380347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034bdd0-9387-4e71-947a-e8acb17b6720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:16.381759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034bdd0-9387-4e71-947a-e8acb17b6720 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:16.387297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 570/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:20:16 2022] GET /identity/v3/role_assignments?user.id=65e74fa4941a484e8bcfb4c386eb77f9&scope.project.id=cdbd5dff18fd46ab808a6248f179f3c9&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:18.302206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ef3c1-6f7f-4f12-b561-aa44d7f3dde0 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:18.302757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ef3c1-6f7f-4f12-b561-aa44d7f3dde0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:18.302757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ef3c1-6f7f-4f12-b561-aa44d7f3dde0 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:18.304072 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 577/1147] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:18.309368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05a6be-6b05-4630-bc55-c6c176d5d6b5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:18.309787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05a6be-6b05-4630-bc55-c6c176d5d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:18.309914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05a6be-6b05-4630-bc55-c6c176d5d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:18.328592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1c05a6be-6b05-4630-bc55-c6c176d5d6b5 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:18.329758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1c05a6be-6b05-4630-bc55-c6c176d5d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:18.396475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 571/1148] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:18.668626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ef3c1-6f7f-4f12-b561-aa44d7f3dde0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:18.700425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:18.771400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ef3c1-6f7f-4f12-b561-aa44d7f3dde0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:18.805955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910e022-6c08-49dc-80cb-5ce44be9bc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:18.806570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910e022-6c08-49dc-80cb-5ce44be9bc64 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:18.806848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910e022-6c08-49dc-80cb-5ce44be9bc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:18.807118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910e022-6c08-49dc-80cb-5ce44be9bc64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:18.817147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910e022-6c08-49dc-80cb-5ce44be9bc64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-b910e022-6c08-49dc-80cb-5ce44be9bc64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:20:18.821876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:20:18.822938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:20:18.823850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 578/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:20:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:20:18.829099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05a6be-6b05-4630-bc55-c6c176d5d6b5 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:18.915897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:18.928126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05a6be-6b05-4630-bc55-c6c176d5d6b5 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:18.962771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:18.963372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:18.963656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:18.963931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:18.964631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:18.964939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:18.966296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:18.975080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 572/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:20:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:20:18.981128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910e022-6c08-49dc-80cb-5ce44be9bc64 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:19.073407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:19.082549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910e022-6c08-49dc-80cb-5ce44be9bc64 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:19.116196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e1fec-705b-41f4-8441-a767f532833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:19.116833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1fec-705b-41f4-8441-a767f532833f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:19.117112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1fec-705b-41f4-8441-a767f532833f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:19.117386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1fec-705b-41f4-8441-a767f532833f None admin] PATH_INFO: `/v3/users/65e74fa4941a484e8bcfb4c386eb77f9` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:19.118500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e1fec-705b-41f4-8441-a767f532833f None admin] RBAC: Authorizing `identity:get_user(user_id=65e74fa4941a484e8bcfb4c386eb77f9)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:19.119929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e1fec-705b-41f4-8441-a767f532833f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:19.121570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 579/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:18 2022] GET /identity/v3/users/65e74fa4941a484e8bcfb4c386eb77f9 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:19.127821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:19.226623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad004af-db15-4536-93aa-a76d939ce34a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:19.260475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514711b-ca99-47ff-ab3d-7ad33de3458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:19.261088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514711b-ca99-47ff-ab3d-7ad33de3458e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:19.261368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514711b-ca99-47ff-ab3d-7ad33de3458e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:19.261636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514711b-ca99-47ff-ab3d-7ad33de3458e None admin] PATH_INFO: `/v3/projects/cdbd5dff18fd46ab808a6248f179f3c9` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:19.262738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514711b-ca99-47ff-ab3d-7ad33de3458e None admin] RBAC: Authorizing `identity:get_project(project_id=cdbd5dff18fd46ab808a6248f179f3c9)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:19.264150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514711b-ca99-47ff-ab3d-7ad33de3458e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:19.266011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 573/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:19 2022] GET /identity/v3/projects/cdbd5dff18fd46ab808a6248f179f3c9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:19.271987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e1fec-705b-41f4-8441-a767f532833f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:19.377521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e1fec-705b-41f4-8441-a767f532833f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:19.414843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb472a4f-9f9a-4527-acd8-d9db41013deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:19.415497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb472a4f-9f9a-4527-acd8-d9db41013deb None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:19.415779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb472a4f-9f9a-4527-acd8-d9db41013deb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:19.416056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb472a4f-9f9a-4527-acd8-d9db41013deb None admin] PATH_INFO: `/v3/projects/cdbd5dff18fd46ab808a6248f179f3c9/users/65e74fa4941a484e8bcfb4c386eb77f9/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:19.418086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb472a4f-9f9a-4527-acd8-d9db41013deb None admin] RBAC: Authorizing `identity:create_grant(project_id=cdbd5dff18fd46ab808a6248f179f3c9, user_id=65e74fa4941a484e8bcfb4c386eb77f9, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:19.419520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb472a4f-9f9a-4527-acd8-d9db41013deb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:19.518855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 580/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:20:19 2022] PUT /identity/v3/projects/cdbd5dff18fd46ab808a6248f179f3c9/users/65e74fa4941a484e8bcfb4c386eb77f9/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:21.496328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c4ec3-0592-4fef-b23d-befab88bac5b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:21.496970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c4ec3-0592-4fef-b23d-befab88bac5b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:21.496970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c4ec3-0592-4fef-b23d-befab88bac5b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:21.498211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 574/1154] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:21.503865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75aa8b-591e-4a9b-9396-ce7b91d67562 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:21.504340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75aa8b-591e-4a9b-9396-ce7b91d67562 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:21.504439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75aa8b-591e-4a9b-9396-ce7b91d67562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:21.523086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7b75aa8b-591e-4a9b-9396-ce7b91d67562 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:21.524308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7b75aa8b-591e-4a9b-9396-ce7b91d67562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:21.539314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:21.592146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 581/1155] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:21.867763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c4ec3-0592-4fef-b23d-befab88bac5b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:21.887458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:21.975375 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c4ec3-0592-4fef-b23d-befab88bac5b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:22.009498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a678c05-b3c1-45e0-90e3-fa4badeb1664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:22.010044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678c05-b3c1-45e0-90e3-fa4badeb1664 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:22.010322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678c05-b3c1-45e0-90e3-fa4badeb1664 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:22.010591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a678c05-b3c1-45e0-90e3-fa4badeb1664 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:22.021081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a678c05-b3c1-45e0-90e3-fa4badeb1664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-3a678c05-b3c1-45e0-90e3-fa4badeb1664 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:20:22.025726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:20:22.026916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:20:22.027962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 575/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:20:22.033421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75aa8b-591e-4a9b-9396-ce7b91d67562 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:22.132412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75aa8b-591e-4a9b-9396-ce7b91d67562 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:22.165520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:22.166117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:22.166399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:22.166674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:22.167337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:22.167640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:22.169068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:22.178020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 582/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:20:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:22.183975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a678c05-b3c1-45e0-90e3-fa4badeb1664 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:22.267059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:22.289244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a678c05-b3c1-45e0-90e3-fa4badeb1664 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:22.323697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5496f-c31a-45da-bd19-44373fbb8530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:22.324349 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5496f-c31a-45da-bd19-44373fbb8530 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:22.324631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5496f-c31a-45da-bd19-44373fbb8530 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:22.324904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5496f-c31a-45da-bd19-44373fbb8530 None admin] PATH_INFO: `/v3/users/65e74fa4941a484e8bcfb4c386eb77f9` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:22.326030 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc5496f-c31a-45da-bd19-44373fbb8530 None admin] RBAC: Authorizing `identity:get_user(user_id=65e74fa4941a484e8bcfb4c386eb77f9)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:22.327542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc5496f-c31a-45da-bd19-44373fbb8530 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:22.329169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 576/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:22 2022] GET /identity/v3/users/65e74fa4941a484e8bcfb4c386eb77f9 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:22.335434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:22.351356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:22.442547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388ea16-ffb3-4f5d-b2bd-033d12fec12c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:22.475586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e15e-71a3-48bc-b2ed-c813a295f248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:22.476201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e15e-71a3-48bc-b2ed-c813a295f248 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:22.476489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e15e-71a3-48bc-b2ed-c813a295f248 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:22.476756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4e15e-71a3-48bc-b2ed-c813a295f248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:22.543551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4e15e-71a3-48bc-b2ed-c813a295f248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:22.545114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4e15e-71a3-48bc-b2ed-c813a295f248 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:22.559570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:22.644380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 583/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:20:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:20:22.650129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5496f-c31a-45da-bd19-44373fbb8530 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:22.754959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5496f-c31a-45da-bd19-44373fbb8530 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:22.789893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243b2f3-f821-4616-acfb-5f08e37e7cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:22.790512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243b2f3-f821-4616-acfb-5f08e37e7cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:22.790795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243b2f3-f821-4616-acfb-5f08e37e7cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:22.791064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243b2f3-f821-4616-acfb-5f08e37e7cf0 None admin] PATH_INFO: `/v3/projects/cdbd5dff18fd46ab808a6248f179f3c9` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:22.792164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243b2f3-f821-4616-acfb-5f08e37e7cf0 None admin] RBAC: Authorizing `identity:get_project(project_id=cdbd5dff18fd46ab808a6248f179f3c9)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:22.793658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243b2f3-f821-4616-acfb-5f08e37e7cf0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:22.795224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 577/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:22 2022] GET /identity/v3/projects/cdbd5dff18fd46ab808a6248f179f3c9 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:22.801859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e15e-71a3-48bc-b2ed-c813a295f248 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:22.904853 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4e15e-71a3-48bc-b2ed-c813a295f248 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:22.918373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:22.940556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94616aff-c26f-407e-8653-ad7c495e5efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:22.941155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616aff-c26f-407e-8653-ad7c495e5efb None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:22.941439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616aff-c26f-407e-8653-ad7c495e5efb None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:22.941716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94616aff-c26f-407e-8653-ad7c495e5efb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:22.942438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94616aff-c26f-407e-8653-ad7c495e5efb None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=cdbd5dff18fd46ab808a6248f179f3c9, user.id=65e74fa4941a484e8bcfb4c386eb77f9) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:22.942749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94616aff-c26f-407e-8653-ad7c495e5efb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:22.944195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94616aff-c26f-407e-8653-ad7c495e5efb None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:22.949849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 584/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:20:22 2022] GET /identity/v3/role_assignments?user.id=65e74fa4941a484e8bcfb4c386eb77f9&scope.project.id=cdbd5dff18fd46ab808a6248f179f3c9&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:24.890176 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009b58-3d08-4a85-b0f9-bc39e9c0dc64 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:24.890702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009b58-3d08-4a85-b0f9-bc39e9c0dc64 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:24.890702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba009b58-3d08-4a85-b0f9-bc39e9c0dc64 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:24.892052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 578/1162] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:24.897476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0ddba-feed-45b8-8238-decbc07bfad3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:24.897895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0ddba-feed-45b8-8238-decbc07bfad3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:24.898035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0ddba-feed-45b8-8238-decbc07bfad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:24.916874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1fc0ddba-feed-45b8-8238-decbc07bfad3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:24.918059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1fc0ddba-feed-45b8-8238-decbc07bfad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:24.986401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 585/1163] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:25.259419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba009b58-3d08-4a85-b0f9-bc39e9c0dc64 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:25.276042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:25.360546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba009b58-3d08-4a85-b0f9-bc39e9c0dc64 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:25.398671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cb625-61ff-43a6-9581-a34a40df3dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:25.399271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cb625-61ff-43a6-9581-a34a40df3dcf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:25.399546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cb625-61ff-43a6-9581-a34a40df3dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:25.399815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cb625-61ff-43a6-9581-a34a40df3dcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:25.466447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765cb625-61ff-43a6-9581-a34a40df3dcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:25.467925 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765cb625-61ff-43a6-9581-a34a40df3dcf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:25.481763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:25.563988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 579/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:20:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:20:25.570110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0ddba-feed-45b8-8238-decbc07bfad3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:25.674632 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0ddba-feed-45b8-8238-decbc07bfad3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:25.681875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:25.708325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eeda3c-0aac-47d5-9e40-418fd9eedca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:25.708944 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeda3c-0aac-47d5-9e40-418fd9eedca4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:25.709311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeda3c-0aac-47d5-9e40-418fd9eedca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:25.709588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eeda3c-0aac-47d5-9e40-418fd9eedca4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:25.716066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eeda3c-0aac-47d5-9e40-418fd9eedca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:25.717539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eeda3c-0aac-47d5-9e40-418fd9eedca4 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a8eeda3c-0aac-47d5-9e40-418fd9eedca4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:20:25.729854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:20:25.731094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:20:25.732267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 586/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:20:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:20:25.737702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cb625-61ff-43a6-9581-a34a40df3dcf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:25.836646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cb625-61ff-43a6-9581-a34a40df3dcf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:25.850698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:25.871511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95aedc-8d53-49ab-a610-645544205e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:25.872116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95aedc-8d53-49ab-a610-645544205e19 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:25.872436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95aedc-8d53-49ab-a610-645544205e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:25.872709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95aedc-8d53-49ab-a610-645544205e19 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:25.873415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95aedc-8d53-49ab-a610-645544205e19 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:25.873847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95aedc-8d53-49ab-a610-645544205e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:25.875794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95aedc-8d53-49ab-a610-645544205e19 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:25.883126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 580/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 13:20:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:27.793863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c28605-b7b4-4770-9be1-5e9b2c6475ff None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:27.794397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c28605-b7b4-4770-9be1-5e9b2c6475ff None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:27.794397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c28605-b7b4-4770-9be1-5e9b2c6475ff None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:27.795902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 587/1167] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:27.801568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a339d-9bd6-4fb0-8cfc-e3290dce7f38 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:27.801970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a339d-9bd6-4fb0-8cfc-e3290dce7f38 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:27.802122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a339d-9bd6-4fb0-8cfc-e3290dce7f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:27.820793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-147a339d-9bd6-4fb0-8cfc-e3290dce7f38 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:27.821983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-147a339d-9bd6-4fb0-8cfc-e3290dce7f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:27.891432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 581/1168] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:28.162361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c28605-b7b4-4770-9be1-5e9b2c6475ff None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:28.265426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c28605-b7b4-4770-9be1-5e9b2c6475ff None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:28.300506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd60025-ee5b-4961-9105-56b99b9b2eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:28.301114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd60025-ee5b-4961-9105-56b99b9b2eba None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:28.301413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd60025-ee5b-4961-9105-56b99b9b2eba None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:28.301706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd60025-ee5b-4961-9105-56b99b9b2eba None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:28.302401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd60025-ee5b-4961-9105-56b99b9b2eba None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:28.303764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd60025-ee5b-4961-9105-56b99b9b2eba None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:28.393467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 588/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Wed Aug 24 13:20:28 2022] POST /identity/v3/domains => generated 267 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:30.397109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a64e-a495-407b-a7f3-1f03f9c0e9f2 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:30.397901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a64e-a495-407b-a7f3-1f03f9c0e9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:30.397901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a64e-a495-407b-a7f3-1f03f9c0e9f2 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:30.399097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 582/1170] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:30.405654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0610cda-d852-445f-bbe7-dedf45643cad None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:30.406209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0610cda-d852-445f-bbe7-dedf45643cad None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:30.406209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0610cda-d852-445f-bbe7-dedf45643cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:30.426783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a0610cda-d852-445f-bbe7-dedf45643cad None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:30.427995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a0610cda-d852-445f-bbe7-dedf45643cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:30.454739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:30.509009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 589/1171] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:30.804699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61a64e-a495-407b-a7f3-1f03f9c0e9f2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:30.923873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61a64e-a495-407b-a7f3-1f03f9c0e9f2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:30.933812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:30.961496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d60c0c-f007-4a1d-85c2-32db622a3f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:30.962231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d60c0c-f007-4a1d-85c2-32db622a3f34 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:30.962732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d60c0c-f007-4a1d-85c2-32db622a3f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:30.963256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d60c0c-f007-4a1d-85c2-32db622a3f34 None admin] PATH_INFO: `/v3/domains/4b1cc8824d8b4a99a64cc7232b418335` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:30.976341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d60c0c-f007-4a1d-85c2-32db622a3f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4b1cc8824d8b4a99a64cc7232b418335)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:30.977797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d60c0c-f007-4a1d-85c2-32db622a3f34 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:30.979581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 583/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Wed Aug 24 13:20:30 2022] GET /identity/v3/domains/4b1cc8824d8b4a99a64cc7232b418335 => generated 267 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:30.985692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0610cda-d852-445f-bbe7-dedf45643cad None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:31.085551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0610cda-d852-445f-bbe7-dedf45643cad None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:31.121541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087214e-c073-401c-a55a-0ed098c13458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:31.122182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087214e-c073-401c-a55a-0ed098c13458 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:31.122459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087214e-c073-401c-a55a-0ed098c13458 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:31.122727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087214e-c073-401c-a55a-0ed098c13458 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:31.123368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087214e-c073-401c-a55a-0ed098c13458 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:31.124859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087214e-c073-401c-a55a-0ed098c13458 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:31.222653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 590/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:20:30 2022] POST /identity/v3/projects => generated 361 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:33.180202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3aeca-c501-4d35-ac62-5a01687bc2f3 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:33.180760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3aeca-c501-4d35-ac62-5a01687bc2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:33.180760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3aeca-c501-4d35-ac62-5a01687bc2f3 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:33.182106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 584/1174] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:33.187474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913ce51-4634-49db-8f56-164f7ccd2693 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:33.187929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913ce51-4634-49db-8f56-164f7ccd2693 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:33.188030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913ce51-4634-49db-8f56-164f7ccd2693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:33.207431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c913ce51-4634-49db-8f56-164f7ccd2693 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:33.208668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c913ce51-4634-49db-8f56-164f7ccd2693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:33.210663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:33.279316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 591/1175] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:33.547980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3aeca-c501-4d35-ac62-5a01687bc2f3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:33.654465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3aeca-c501-4d35-ac62-5a01687bc2f3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:33.688241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffc6ed-f403-4db2-822c-37677c66955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:33.688864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc6ed-f403-4db2-822c-37677c66955e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:33.689139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc6ed-f403-4db2-822c-37677c66955e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:33.689408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc6ed-f403-4db2-822c-37677c66955e None admin] PATH_INFO: `/v3/domains/4b1cc8824d8b4a99a64cc7232b418335` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:33.690512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffc6ed-f403-4db2-822c-37677c66955e None admin] RBAC: Authorizing `identity:get_domain(domain_id=4b1cc8824d8b4a99a64cc7232b418335)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:33.691933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffc6ed-f403-4db2-822c-37677c66955e None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:33.693507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 585/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Wed Aug 24 13:20:33 2022] GET /identity/v3/domains/4b1cc8824d8b4a99a64cc7232b418335 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:33.699369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913ce51-4634-49db-8f56-164f7ccd2693 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:33.807005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913ce51-4634-49db-8f56-164f7ccd2693 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:33.841260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef93db-d808-4c62-891d-243b6c6f2698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:33.841848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef93db-d808-4c62-891d-243b6c6f2698 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:33.842124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef93db-d808-4c62-891d-243b6c6f2698 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:33.843221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef93db-d808-4c62-891d-243b6c6f2698 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:33.843976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef93db-d808-4c62-891d-243b6c6f2698 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:33.845418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef93db-d808-4c62-891d-243b6c6f2698 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:33.855139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:33.969622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 592/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:20:33 2022] POST /identity/v3/users => generated 311 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:20:35.988066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea710b-9bdb-4609-ad89-3e28cdf80ca2 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.988692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea710b-9bdb-4609-ad89-3e28cdf80ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.988692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea710b-9bdb-4609-ad89-3e28cdf80ca2 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:35.989971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 586/1178] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:35.995105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f2396-c544-4054-83e8-7c1f5fc3e478 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:35.995507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f2396-c544-4054-83e8-7c1f5fc3e478 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:35.995659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f2396-c544-4054-83e8-7c1f5fc3e478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.015078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-423f2396-c544-4054-83e8-7c1f5fc3e478 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:36.016404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-423f2396-c544-4054-83e8-7c1f5fc3e478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:36.083512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 593/1179] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:36.351154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea710b-9bdb-4609-ad89-3e28cdf80ca2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:36.388038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:36.453437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea710b-9bdb-4609-ad89-3e28cdf80ca2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:36.486274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8f866-2314-4a7f-ae1c-1584d0d58a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:36.486886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8f866-2314-4a7f-ae1c-1584d0d58a2b None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.487163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8f866-2314-4a7f-ae1c-1584d0d58a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.487429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8f866-2314-4a7f-ae1c-1584d0d58a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.497699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8f866-2314-4a7f-ae1c-1584d0d58a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-02e8f866-2314-4a7f-ae1c-1584d0d58a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:20:36.503092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:20:36.504003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:20:36.505103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 587/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:20:36.510443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f2396-c544-4054-83e8-7c1f5fc3e478 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:36.611819 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f2396-c544-4054-83e8-7c1f5fc3e478 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:36.634895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:36.645489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:36.653254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.653254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.653254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.653254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:36.653254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:36.653254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:36.657436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 594/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:36.663507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8f866-2314-4a7f-ae1c-1584d0d58a2b None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:36.760277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:36.766587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8f866-2314-4a7f-ae1c-1584d0d58a2b None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:36.805847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b05f4-1db0-4115-9a92-f4eaf8ac950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:36.806480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b05f4-1db0-4115-9a92-f4eaf8ac950c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.806889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b05f4-1db0-4115-9a92-f4eaf8ac950c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.807179 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b05f4-1db0-4115-9a92-f4eaf8ac950c None admin] PATH_INFO: `/v3/users/bbd86d6f393142418093be993b916afe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:36.821200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b05f4-1db0-4115-9a92-f4eaf8ac950c None admin] RBAC: Authorizing `identity:get_user(user_id=bbd86d6f393142418093be993b916afe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:36.822633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b05f4-1db0-4115-9a92-f4eaf8ac950c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:36.824280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 588/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:36 2022] GET /identity/v3/users/bbd86d6f393142418093be993b916afe => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:36.831416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:36.929282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c93af-0cab-47e2-9925-10be703ceb51 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:36.962205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46745ae3-3a11-470d-bb47-20aed288c656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:36.962793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46745ae3-3a11-470d-bb47-20aed288c656 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:36.963069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46745ae3-3a11-470d-bb47-20aed288c656 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:36.963336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46745ae3-3a11-470d-bb47-20aed288c656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:37.016709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:37.029118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46745ae3-3a11-470d-bb47-20aed288c656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:37.030708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46745ae3-3a11-470d-bb47-20aed288c656 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:37.142434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 595/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:20:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 13:20:37.150068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b05f4-1db0-4115-9a92-f4eaf8ac950c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:37.251691 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b05f4-1db0-4115-9a92-f4eaf8ac950c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:37.286012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0c45c-c23d-49cd-93ea-596e6eaa0faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:37.286621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c45c-c23d-49cd-93ea-596e6eaa0faa None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:37.286902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c45c-c23d-49cd-93ea-596e6eaa0faa None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:37.287173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0c45c-c23d-49cd-93ea-596e6eaa0faa None admin] PATH_INFO: `/v3/projects/c5b40ad650564fba9f81d9fbcde6dfef` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:37.288302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0c45c-c23d-49cd-93ea-596e6eaa0faa None admin] RBAC: Authorizing `identity:get_project(project_id=c5b40ad650564fba9f81d9fbcde6dfef)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:37.289717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0c45c-c23d-49cd-93ea-596e6eaa0faa None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:37.291322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 589/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:37 2022] GET /identity/v3/projects/c5b40ad650564fba9f81d9fbcde6dfef => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:37.298800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46745ae3-3a11-470d-bb47-20aed288c656 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:37.382902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:37.397752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46745ae3-3a11-470d-bb47-20aed288c656 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:37.434418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf453c9-b74a-4a5b-badc-aec0a055835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:37.435013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf453c9-b74a-4a5b-badc-aec0a055835c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:37.435295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf453c9-b74a-4a5b-badc-aec0a055835c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:37.435568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf453c9-b74a-4a5b-badc-aec0a055835c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:37.437289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf453c9-b74a-4a5b-badc-aec0a055835c None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=c5b40ad650564fba9f81d9fbcde6dfef, user.id=bbd86d6f393142418093be993b916afe) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:37.437602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf453c9-b74a-4a5b-badc-aec0a055835c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:37.439037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf453c9-b74a-4a5b-badc-aec0a055835c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:37.444447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 596/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:20:37 2022] GET /identity/v3/role_assignments?user.id=bbd86d6f393142418093be993b916afe&scope.project.id=c5b40ad650564fba9f81d9fbcde6dfef&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:39.418037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6733c2-9c39-415c-a407-2d44162f9a76 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:39.418568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6733c2-9c39-415c-a407-2d44162f9a76 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:39.418568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6733c2-9c39-415c-a407-2d44162f9a76 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:39.419861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 590/1186] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:39.426070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75c43d-ed5e-4275-a716-8dabbb57439d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:39.426481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75c43d-ed5e-4275-a716-8dabbb57439d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:39.426657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75c43d-ed5e-4275-a716-8dabbb57439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:39.446521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8f75c43d-ed5e-4275-a716-8dabbb57439d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:39.447716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8f75c43d-ed5e-4275-a716-8dabbb57439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:39.516298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 597/1187] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:39.790152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6733c2-9c39-415c-a407-2d44162f9a76 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:39.815493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:39.892057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6733c2-9c39-415c-a407-2d44162f9a76 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:39.925541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5cdc8-9837-4cf9-a3bd-3face2722974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:39.926213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5cdc8-9837-4cf9-a3bd-3face2722974 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:39.926500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5cdc8-9837-4cf9-a3bd-3face2722974 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:39.926774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5cdc8-9837-4cf9-a3bd-3face2722974 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:39.937356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5cdc8-9837-4cf9-a3bd-3face2722974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-78a5cdc8-9837-4cf9-a3bd-3face2722974 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:20:39.944868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:20:39.946491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:20:39.949081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 591/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:20:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:20:39.953714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75c43d-ed5e-4275-a716-8dabbb57439d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.038021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:40.057039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75c43d-ed5e-4275-a716-8dabbb57439d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.092077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.093642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.093928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.094205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.094875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:40.095183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.096584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.104652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 598/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:20:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:40.110626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5cdc8-9837-4cf9-a3bd-3face2722974 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.191163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:40.209403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5cdc8-9837-4cf9-a3bd-3face2722974 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.243192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6abd2-3192-4b15-93f8-5597de9e703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.243814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6abd2-3192-4b15-93f8-5597de9e703c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.244085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6abd2-3192-4b15-93f8-5597de9e703c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.244384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6abd2-3192-4b15-93f8-5597de9e703c None admin] PATH_INFO: `/v3/users/bbd86d6f393142418093be993b916afe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.245486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6abd2-3192-4b15-93f8-5597de9e703c None admin] RBAC: Authorizing `identity:get_user(user_id=bbd86d6f393142418093be993b916afe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.246944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6abd2-3192-4b15-93f8-5597de9e703c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.248651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 592/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:40 2022] GET /identity/v3/users/bbd86d6f393142418093be993b916afe => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:40.254593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.355412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331b172-3d9f-4525-bf26-4f3d2eb350e9 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.389631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d288e-c4d0-4528-8187-4c747d702783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.390309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d288e-c4d0-4528-8187-4c747d702783 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.390708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d288e-c4d0-4528-8187-4c747d702783 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.390989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d288e-c4d0-4528-8187-4c747d702783 None admin] PATH_INFO: `/v3/projects/c5b40ad650564fba9f81d9fbcde6dfef` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.392125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d288e-c4d0-4528-8187-4c747d702783 None admin] RBAC: Authorizing `identity:get_project(project_id=c5b40ad650564fba9f81d9fbcde6dfef)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.393590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d288e-c4d0-4528-8187-4c747d702783 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.395231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 599/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:40 2022] GET /identity/v3/projects/c5b40ad650564fba9f81d9fbcde6dfef => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:40.401479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6abd2-3192-4b15-93f8-5597de9e703c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:40.500285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6abd2-3192-4b15-93f8-5597de9e703c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:40.533264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1e0cd-e896-4be2-83f5-08a9a8714148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:40.533901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e0cd-e896-4be2-83f5-08a9a8714148 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:40.534180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e0cd-e896-4be2-83f5-08a9a8714148 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:40.534451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e0cd-e896-4be2-83f5-08a9a8714148 None admin] PATH_INFO: `/v3/projects/c5b40ad650564fba9f81d9fbcde6dfef/users/bbd86d6f393142418093be993b916afe/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:40.536362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1e0cd-e896-4be2-83f5-08a9a8714148 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5b40ad650564fba9f81d9fbcde6dfef, user_id=bbd86d6f393142418093be993b916afe, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:40.537779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1e0cd-e896-4be2-83f5-08a9a8714148 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:40.644680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 593/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:20:40 2022] PUT /identity/v3/projects/c5b40ad650564fba9f81d9fbcde6dfef/users/bbd86d6f393142418093be993b916afe/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:20:42.529528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca42cee-4a37-4767-9244-ce192aa3110f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:42.530063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca42cee-4a37-4767-9244-ce192aa3110f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:42.530063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca42cee-4a37-4767-9244-ce192aa3110f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:42.531427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 600/1193] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:20:42.538665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29bed4-bbae-4ec7-b51e-239ef49e724f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:42.539150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29bed4-bbae-4ec7-b51e-239ef49e724f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:42.539245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29bed4-bbae-4ec7-b51e-239ef49e724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:42.557672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4d29bed4-bbae-4ec7-b51e-239ef49e724f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:20:42.558869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4d29bed4-bbae-4ec7-b51e-239ef49e724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:20:42.562646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:42.631604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 594/1194] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:20:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:20:42.913201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca42cee-4a37-4767-9244-ce192aa3110f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:42.926330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:43.016660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca42cee-4a37-4767-9244-ce192aa3110f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:43.051304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8825ee-964b-49ad-b7a7-82e680e7fbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:43.051911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8825ee-964b-49ad-b7a7-82e680e7fbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:43.052211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8825ee-964b-49ad-b7a7-82e680e7fbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:43.052487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8825ee-964b-49ad-b7a7-82e680e7fbd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:43.062848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8825ee-964b-49ad-b7a7-82e680e7fbd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7c8825ee-964b-49ad-b7a7-82e680e7fbd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:20:43.067530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:20:43.068672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:20:43.069912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 601/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:20:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:20:43.074820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29bed4-bbae-4ec7-b51e-239ef49e724f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:43.172632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29bed4-bbae-4ec7-b51e-239ef49e724f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:43.207142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:43.207741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:43.208023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:43.208326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:43.208994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:43.209303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:43.210660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:43.220254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 595/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:20:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:43.226398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8825ee-964b-49ad-b7a7-82e680e7fbd4 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:43.296739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:43.325993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8825ee-964b-49ad-b7a7-82e680e7fbd4 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:43.360793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233fa2-ed01-4055-8728-6e3fb80b6733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:43.361411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233fa2-ed01-4055-8728-6e3fb80b6733 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:43.361691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233fa2-ed01-4055-8728-6e3fb80b6733 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:43.361964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66233fa2-ed01-4055-8728-6e3fb80b6733 None admin] PATH_INFO: `/v3/users/bbd86d6f393142418093be993b916afe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:43.363076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66233fa2-ed01-4055-8728-6e3fb80b6733 None admin] RBAC: Authorizing `identity:get_user(user_id=bbd86d6f393142418093be993b916afe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:43.364598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66233fa2-ed01-4055-8728-6e3fb80b6733 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:43.366173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 602/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 13:20:43 2022] GET /identity/v3/users/bbd86d6f393142418093be993b916afe => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:43.372337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:43.375894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:43.474189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed6c91-9dd2-4607-9912-dc2136b9383c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:43.506970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa779c-f67d-44b4-9579-0ac9782dd697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:43.507556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa779c-f67d-44b4-9579-0ac9782dd697 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:43.507830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa779c-f67d-44b4-9579-0ac9782dd697 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:43.508096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa779c-f67d-44b4-9579-0ac9782dd697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:43.572885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa779c-f67d-44b4-9579-0ac9782dd697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:43.574374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa779c-f67d-44b4-9579-0ac9782dd697 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:43.576137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:43.671515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 596/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:20:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:20:43.677740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233fa2-ed01-4055-8728-6e3fb80b6733 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:43.778979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66233fa2-ed01-4055-8728-6e3fb80b6733 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:43.813748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4686f7-f3dc-4e12-ba09-dbb657ba133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:43.814362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4686f7-f3dc-4e12-ba09-dbb657ba133f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:43.814641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4686f7-f3dc-4e12-ba09-dbb657ba133f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:43.814916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4686f7-f3dc-4e12-ba09-dbb657ba133f None admin] PATH_INFO: `/v3/projects/c5b40ad650564fba9f81d9fbcde6dfef` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:43.816053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4686f7-f3dc-4e12-ba09-dbb657ba133f None admin] RBAC: Authorizing `identity:get_project(project_id=c5b40ad650564fba9f81d9fbcde6dfef)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:43.817555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4686f7-f3dc-4e12-ba09-dbb657ba133f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:43.819176 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 603/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 13:20:43 2022] GET /identity/v3/projects/c5b40ad650564fba9f81d9fbcde6dfef => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:43.825415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa779c-f67d-44b4-9579-0ac9782dd697 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:20:43.923405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa779c-f67d-44b4-9579-0ac9782dd697 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:20:43.925192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:43.958137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c3f97-8377-4e66-95a1-36efcab2e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:20:43.958743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c3f97-8377-4e66-95a1-36efcab2e068 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:20:43.959022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c3f97-8377-4e66-95a1-36efcab2e068 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:20:43.959295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c3f97-8377-4e66-95a1-36efcab2e068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:20:43.960015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c3f97-8377-4e66-95a1-36efcab2e068 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=c5b40ad650564fba9f81d9fbcde6dfef, user.id=bbd86d6f393142418093be993b916afe) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:20:43.960353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c3f97-8377-4e66-95a1-36efcab2e068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:20:43.961767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c3f97-8377-4e66-95a1-36efcab2e068 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:20:43.967944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 597/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:20:43 2022] GET /identity/v3/role_assignments?user.id=bbd86d6f393142418093be993b916afe&scope.project.id=c5b40ad650564fba9f81d9fbcde6dfef&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:20:53.808856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:20:53.974846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=17 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:02.040536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932299e-ba1a-4501-beaa-d64e7677e0ee None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:02.041136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932299e-ba1a-4501-beaa-d64e7677e0ee None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:02.041136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932299e-ba1a-4501-beaa-d64e7677e0ee None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:02.042429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 604/1201] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:22:02.047960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c02a89-7206-4935-9748-4473cb36480e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:02.049504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c02a89-7206-4935-9748-4473cb36480e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:02.049504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c02a89-7206-4935-9748-4473cb36480e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:02.067392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-31c02a89-7206-4935-9748-4473cb36480e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:02.068602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-31c02a89-7206-4935-9748-4473cb36480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:02.133859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 598/1202] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:02.419987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932299e-ba1a-4501-beaa-d64e7677e0ee None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:02.525825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932299e-ba1a-4501-beaa-d64e7677e0ee None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:02.559649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:02.560275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:02.560557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:02.560822 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:02.571139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:02.572685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:02.587317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:02.588592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:02.589601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:02.589601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:02.589601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:02.589601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 605/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:02.594700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c02a89-7206-4935-9748-4473cb36480e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:02.695978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c02a89-7206-4935-9748-4473cb36480e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:02.725825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:02.731500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c1bf8-e694-4e08-8a13-85fabbed78a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:02.732102 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c1bf8-e694-4e08-8a13-85fabbed78a9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:02.732413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c1bf8-e694-4e08-8a13-85fabbed78a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:02.732685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c1bf8-e694-4e08-8a13-85fabbed78a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:02.733324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c1bf8-e694-4e08-8a13-85fabbed78a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:02.733626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c1bf8-e694-4e08-8a13-85fabbed78a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:02.734993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c1bf8-e694-4e08-8a13-85fabbed78a9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:02.746016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 599/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:02.751954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:02.805886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:02.856889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26afa-4901-4c15-becf-6814750145bf None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:02.891122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1580491-a763-4302-a304-79e269b88a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:02.891701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1580491-a763-4302-a304-79e269b88a5e None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:02.891978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1580491-a763-4302-a304-79e269b88a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:02.892345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1580491-a763-4302-a304-79e269b88a5e None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:02.892986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1580491-a763-4302-a304-79e269b88a5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:02.896428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1580491-a763-4302-a304-79e269b88a5e None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:03.047459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 606/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 13:22:02 2022] POST /identity/v3/users => generated 251 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:22:04.983001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f076-8758-446c-9f93-d2a1d3aee1ef None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:04.983559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f076-8758-446c-9f93-d2a1d3aee1ef None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:04.983559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f076-8758-446c-9f93-d2a1d3aee1ef None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:04.984897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 600/1206] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:04.990961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3069bf-7379-4105-9fa6-414bbd17840d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:04.991376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3069bf-7379-4105-9fa6-414bbd17840d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:04.991519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3069bf-7379-4105-9fa6-414bbd17840d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:05.009837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ae3069bf-7379-4105-9fa6-414bbd17840d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:05.011076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ae3069bf-7379-4105-9fa6-414bbd17840d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:05.080954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 607/1207] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:05.347267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f076-8758-446c-9f93-d2a1d3aee1ef None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:05.450825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f076-8758-446c-9f93-d2a1d3aee1ef None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:05.485723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41076e60-ec9f-455d-bfe0-00c4a302377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:05.486353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41076e60-ec9f-455d-bfe0-00c4a302377e None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:05.486637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41076e60-ec9f-455d-bfe0-00c4a302377e None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:05.486919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41076e60-ec9f-455d-bfe0-00c4a302377e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:05.497617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41076e60-ec9f-455d-bfe0-00c4a302377e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-41076e60-ec9f-455d-bfe0-00c4a302377e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:22:05.502513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:05.504101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:05.505239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 601/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:22:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:22:05.509718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3069bf-7379-4105-9fa6-414bbd17840d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:05.551684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:05.612660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3069bf-7379-4105-9fa6-414bbd17840d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:05.648782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:05.649419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:05.649703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:05.649975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:05.650642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:05.650947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:05.652473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:05.660910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 608/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:22:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:05.666682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41076e60-ec9f-455d-bfe0-00c4a302377e None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:05.717887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:05.770900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41076e60-ec9f-455d-bfe0-00c4a302377e None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:05.803407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:05.804097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:05.804405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:05.804676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:05.814839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:05.817052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:05.832357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:05.833388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:05.834501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:05.834501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:05.834501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:05.834501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 602/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:05.839587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:05.938803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:05.942178 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f673867-9070-4543-8822-6f16a4e31ddf None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:05.974508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:05.975113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:05.975384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:05.975651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:05.976315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:05.976612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:05.978016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:05.991083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 609/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:05.996992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:06.104220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e107-c0ec-46e6-80d1-238cebd55e0a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:06.105777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:06.139259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:06.146036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:06.146036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:06.146036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:06.147993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:06.149458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:22:06.160738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:06.161917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:06.163061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 603/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:06.167843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:06.271388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906d672-d7d9-403c-8567-d4d706895f40 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:06.306023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:06.306623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:06.306910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:06.307181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:06.316156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:06.316494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:06.317942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:06.335791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 610/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 13:22:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:06.341601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:06.443902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf88db7-7b61-47c5-bb40-873aacdc29d7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:06.485776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588cecd-6f10-4693-9464-40f7036d9113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:06.486383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588cecd-6f10-4693-9464-40f7036d9113 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:06.486664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588cecd-6f10-4693-9464-40f7036d9113 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:06.486935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588cecd-6f10-4693-9464-40f7036d9113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:06.516832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:06.555371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588cecd-6f10-4693-9464-40f7036d9113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:06.556780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588cecd-6f10-4693-9464-40f7036d9113 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:06.657091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 604/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:22:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:22:06.663727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:06.683668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:06.761697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86950ab0-569b-4b0e-90ee-039a4a24d40b None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:06.796063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:06.796715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:06.796994 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:06.797265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:06.807365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:06.808790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:06.824144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:06.825340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:06.826230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:06.826230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:06.826230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:06.826230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 611/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:06.831793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588cecd-6f10-4693-9464-40f7036d9113 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:06.892199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:06.931322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588cecd-6f10-4693-9464-40f7036d9113 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:06.964800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:06.965571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:06.965922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:06.966224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:06.966866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:06.967171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:06.968562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:06.980485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 605/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:06.986183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:07.063221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:07.095541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c745122-a218-4152-8332-11c7b81a4f49 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:07.131429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:07.132048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:07.132365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:07.132763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:07.139449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:07.140916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:22:07.153139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:07.154252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:07.155548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 612/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:07.160552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:07.263139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70805e-98bb-4c08-8856-45c17bd3a4e8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:07.286323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:07.297521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a623a7-e7bd-4925-84ba-66c4c8aea0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:07.298119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a623a7-e7bd-4925-84ba-66c4c8aea0de None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:07.298399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a623a7-e7bd-4925-84ba-66c4c8aea0de None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:07.298670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a623a7-e7bd-4925-84ba-66c4c8aea0de None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:07.300285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a623a7-e7bd-4925-84ba-66c4c8aea0de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:07.300612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a623a7-e7bd-4925-84ba-66c4c8aea0de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:07.302072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a623a7-e7bd-4925-84ba-66c4c8aea0de None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:07.313683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 606/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:07.319923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:07.427573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a4012-9b6d-43dd-bb65-da3e349f38b0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:07.456350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:07.462432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a72730-7294-4f9a-95c2-6c1845ee7843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:07.463072 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72730-7294-4f9a-95c2-6c1845ee7843 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:07.463353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72730-7294-4f9a-95c2-6c1845ee7843 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:07.463628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72730-7294-4f9a-95c2-6c1845ee7843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:07.464380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a72730-7294-4f9a-95c2-6c1845ee7843 None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=ea29f79ed2af4341ab6f5ba50a47d8bf) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:07.464694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a72730-7294-4f9a-95c2-6c1845ee7843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:07.466121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a72730-7294-4f9a-95c2-6c1845ee7843 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:07.472774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 613/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:22:07 2022] GET /identity/v3/role_assignments?user.id=ea29f79ed2af4341ab6f5ba50a47d8bf&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:09.446897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac32f1-651d-4a8f-8ffa-c1af62aa76de None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:09.447424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac32f1-651d-4a8f-8ffa-c1af62aa76de None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:09.447531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac32f1-651d-4a8f-8ffa-c1af62aa76de None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:09.449104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 607/1220] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:09.454099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c0117-37c7-4d18-8f32-9857a123390f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:09.454612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c0117-37c7-4d18-8f32-9857a123390f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:09.454714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c0117-37c7-4d18-8f32-9857a123390f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:09.473946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-792c0117-37c7-4d18-8f32-9857a123390f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:09.475152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-792c0117-37c7-4d18-8f32-9857a123390f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:09.543145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 614/1221] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:09.827342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac32f1-651d-4a8f-8ffa-c1af62aa76de None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:09.934099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac32f1-651d-4a8f-8ffa-c1af62aa76de None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:09.968231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253a0c-aa59-45ec-819f-68ae662b08d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:09.968857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253a0c-aa59-45ec-819f-68ae662b08d7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:09.969137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253a0c-aa59-45ec-819f-68ae662b08d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:09.969406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253a0c-aa59-45ec-819f-68ae662b08d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:09.979679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8253a0c-aa59-45ec-819f-68ae662b08d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-f8253a0c-aa59-45ec-819f-68ae662b08d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:22:09.984488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:09.985564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:09.986402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 608/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:22:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 13:22:09.992434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c0117-37c7-4d18-8f32-9857a123390f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:10.093298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c0117-37c7-4d18-8f32-9857a123390f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:10.121608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:10.137501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:10.138177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:10.138470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:10.138769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:10.139455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:10.139762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:10.141187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:10.149790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 615/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:22:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:10.155642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253a0c-aa59-45ec-819f-68ae662b08d7 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:10.201253 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:10.257149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253a0c-aa59-45ec-819f-68ae662b08d7 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:10.290406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:10.291022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:10.291306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:10.291574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:10.302555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:10.303988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:10.318476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:10.319628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:10.320970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:10.320970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:10.320970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:10.320970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 609/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:10.325653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:10.436963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0fcf07-e8d1-4797-82bd-c7fb504dcfe0 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:10.474213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:10.475090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:10.475586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:10.476071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:10.476897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:10.477477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:10.479599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:10.493428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 616/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:10.499238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:10.593668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:10.600968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4612678c-0929-4c36-bb07-91af91f4114f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:10.633316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:10.633935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:10.634212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:10.634482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:10.641740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:10.643730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:22:10.654615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:10.655746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:10.656793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 610/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:10.662508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:10.710302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:10.766360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355905d-80a8-44aa-97e0-da6fcd1638c3 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:10.802728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:10.803329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:10.803728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:10.804008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:10.804774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:10.805082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:10.806512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:10.824705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 617/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 13:22:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:10.832912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:10.934597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c83ca-db59-412b-87aa-7f1090c6765a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:10.969837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:10.970383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:10.970665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:10.970979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:10.984128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:10.985603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:10.999677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:11.005150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:11.006347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:11.006347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:11.006347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:11.006347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 611/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:11.007491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:11.109162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136937a-1860-4dea-afc9-48c1d0ef502c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:11.111438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:11.144375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:11.144971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:11.145244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:11.145517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:11.146159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:11.146460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:11.147823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:11.159121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 618/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:11.165011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:11.175053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:11.264362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70b9a6-17d9-4612-8dfb-cec07f7be9c0 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:11.298159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:11.301548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:11.301548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:11.301548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:11.306862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:11.308412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:22:11.320347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:11.321481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:11.322601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 612/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:22:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:11.327722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:11.430261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a53e5b-6b90-4c12-adac-35e0d9cf2cd1 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:11.464839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae08df0-56cd-4832-8668-5a3c0e1cb06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:11.465440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08df0-56cd-4832-8668-5a3c0e1cb06b None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:11.465722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08df0-56cd-4832-8668-5a3c0e1cb06b None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:11.465998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae08df0-56cd-4832-8668-5a3c0e1cb06b None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:11.466690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae08df0-56cd-4832-8668-5a3c0e1cb06b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:11.466995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae08df0-56cd-4832-8668-5a3c0e1cb06b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:11.468559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae08df0-56cd-4832-8668-5a3c0e1cb06b None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:11.480512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 619/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:22:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:11.486722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:11.555704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:11.590372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d53088d-246b-40e9-9621-0cc6fa4b951f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:11.624089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcc371-f8d5-419f-8bf4-b984d057f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:11.624781 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcc371-f8d5-419f-8bf4-b984d057f600 None admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:11.625063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcc371-f8d5-419f-8bf4-b984d057f600 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:11.625344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcc371-f8d5-419f-8bf4-b984d057f600 None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/ea29f79ed2af4341ab6f5ba50a47d8bf/roles/8760c26505544fdda1848788d4df9b5a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:11.639845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcc371-f8d5-419f-8bf4-b984d057f600 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=ea29f79ed2af4341ab6f5ba50a47d8bf, role_id=8760c26505544fdda1848788d4df9b5a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:11.641320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcc371-f8d5-419f-8bf4-b984d057f600 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:11.721287 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 613/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:22:11 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/ea29f79ed2af4341ab6f5ba50a47d8bf/roles/8760c26505544fdda1848788d4df9b5a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:22:13.689489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d96f1-9c83-41e8-af92-0ec862b874e9 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:13.690020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d96f1-9c83-41e8-af92-0ec862b874e9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:13.690020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d96f1-9c83-41e8-af92-0ec862b874e9 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:13.692110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 620/1233] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:13.697488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668990-1f98-4291-bb6f-29772ce78295 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:13.697933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668990-1f98-4291-bb6f-29772ce78295 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:13.698037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668990-1f98-4291-bb6f-29772ce78295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:13.716788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-34668990-1f98-4291-bb6f-29772ce78295 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:13.717966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-34668990-1f98-4291-bb6f-29772ce78295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:13.789755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 614/1234] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:14.066340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d96f1-9c83-41e8-af92-0ec862b874e9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:14.091402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:14.177984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d96f1-9c83-41e8-af92-0ec862b874e9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:14.211637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824e4c3-d505-47a3-bf26-a809cdd51cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:14.212286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824e4c3-d505-47a3-bf26-a809cdd51cdd None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:14.227764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824e4c3-d505-47a3-bf26-a809cdd51cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:14.227764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824e4c3-d505-47a3-bf26-a809cdd51cdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:14.227764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824e4c3-d505-47a3-bf26-a809cdd51cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-9824e4c3-d505-47a3-bf26-a809cdd51cdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:22:14.228034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:14.232491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:14.233577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 621/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 13:22:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:14.235353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34668990-1f98-4291-bb6f-29772ce78295 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:14.281806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:14.336433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34668990-1f98-4291-bb6f-29772ce78295 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:14.373473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:14.374085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:14.374367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:14.374637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:14.375372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] RBAC: Adding query filter params (name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:14.375675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:14.377151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:14.386785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 615/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 13:22:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:14.393088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824e4c3-d505-47a3-bf26-a809cdd51cdd None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:14.482789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:14.498883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824e4c3-d505-47a3-bf26-a809cdd51cdd None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:14.540366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:14.540997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:14.541281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:14.541557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:14.552009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:14.553489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:14.568607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:14.569743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:14.571014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:14.571014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:14.571014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:14.571014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 622/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:14.575835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:14.678007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f1a17-7975-4d63-9960-99d934ab0555 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:14.686914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:14.713294 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:14.713937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:14.714209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:14.714482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:14.715124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:14.715427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:14.716828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:14.728841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 616/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:14.734693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:14.839209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faca01a-ffc3-4cad-beba-40c6c4eb6ce6 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:14.875248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:14.875870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:14.876151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:14.876459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:14.879194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:14.885135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:14.886587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:22:14.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:14.898473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:14.899499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 623/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:14.904368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:15.004647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c928b73-8c88-4918-a57c-ca75470dfc64 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:15.038526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:15.039115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:15.039392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:15.039663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:15.040406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:15.040710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:15.042133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:15.059379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 617/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 13:22:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:15.065357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:15.171814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feea2b-95c9-4ac9-9d9c-084ebc754249 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:15.205932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9328d8b-fc18-40d9-b585-b46cbe23819f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:15.207221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328d8b-fc18-40d9-b585-b46cbe23819f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:15.207692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328d8b-fc18-40d9-b585-b46cbe23819f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:15.207973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328d8b-fc18-40d9-b585-b46cbe23819f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:15.270338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:15.276994 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9328d8b-fc18-40d9-b585-b46cbe23819f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:15.278436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9328d8b-fc18-40d9-b585-b46cbe23819f None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:15.376076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 624/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:22:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:22:15.383223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:15.411688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:15.486018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629818fe-2bb7-4941-ac8a-94af74d45cd9 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:15.521140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:15.521697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:15.521980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:15.522252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:15.532026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:15.533498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:15.549115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:15.550258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:15.551313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:15.551313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:15.551313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:15.551313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 618/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:15.556269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9328d8b-fc18-40d9-b585-b46cbe23819f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:15.651459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:15.657291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9328d8b-fc18-40d9-b585-b46cbe23819f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:15.692331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:15.692933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:15.693214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:15.693489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:15.694131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:15.694433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:15.695799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:15.707087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 625/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:15.713068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:15.792453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:15.814268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021304eb-5c8a-42b6-87a1-1a73cdcf650f None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:15.848212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:15.848849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:15.849166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:15.849446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:15.856063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:15.857600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:22:15.869745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:15.870762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:15.871884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 619/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:22:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:15.876883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:15.980393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05344d1-4e88-44f8-9090-62ec5a2a0a45 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:16.014613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85366efa-322a-40d0-8257-711df9c466b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:16.015206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366efa-322a-40d0-8257-711df9c466b8 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:16.015486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366efa-322a-40d0-8257-711df9c466b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:16.015758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85366efa-322a-40d0-8257-711df9c466b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:16.016474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85366efa-322a-40d0-8257-711df9c466b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:16.016780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85366efa-322a-40d0-8257-711df9c466b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:16.018211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85366efa-322a-40d0-8257-711df9c466b8 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:16.030197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 626/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:22:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:16.037169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:16.139684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1cf5e-446a-43ff-9db5-91334016fa05 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:16.173093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b2a00-c886-4499-a104-6f1e60e44a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:16.173694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2a00-c886-4499-a104-6f1e60e44a5f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:16.174230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2a00-c886-4499-a104-6f1e60e44a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:16.174503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2a00-c886-4499-a104-6f1e60e44a5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:16.175816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b2a00-c886-4499-a104-6f1e60e44a5f None admin] RBAC: Adding query filter params (role.id=8760c26505544fdda1848788d4df9b5a, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=ea29f79ed2af4341ab6f5ba50a47d8bf) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:16.176120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b2a00-c886-4499-a104-6f1e60e44a5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:16.177609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b2a00-c886-4499-a104-6f1e60e44a5f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:16.183833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 620/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:22:16 2022] GET /identity/v3/role_assignments?user.id=ea29f79ed2af4341ab6f5ba50a47d8bf&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=8760c26505544fdda1848788d4df9b5a => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:18.127506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd838af-8b88-474d-b87b-2e1028e2b467 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:18.128668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd838af-8b88-474d-b87b-2e1028e2b467 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:18.128668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd838af-8b88-474d-b87b-2e1028e2b467 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:18.130114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 627/1247] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:18.135192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195931c0-2aa6-48c0-9c5b-c8b37baa07e2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:18.135653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195931c0-2aa6-48c0-9c5b-c8b37baa07e2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:18.135792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195931c0-2aa6-48c0-9c5b-c8b37baa07e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:18.142080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:18.154488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-195931c0-2aa6-48c0-9c5b-c8b37baa07e2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:18.155672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-195931c0-2aa6-48c0-9c5b-c8b37baa07e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:18.224262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 621/1248] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:18.505992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd838af-8b88-474d-b87b-2e1028e2b467 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:18.529318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:18.618317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd838af-8b88-474d-b87b-2e1028e2b467 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:18.653008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32a092-88a4-4c6b-a227-b4d79866a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:18.653666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32a092-88a4-4c6b-a227-b4d79866a58f None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:18.653945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32a092-88a4-4c6b-a227-b4d79866a58f None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:18.654214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32a092-88a4-4c6b-a227-b4d79866a58f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:18.665030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32a092-88a4-4c6b-a227-b4d79866a58f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-3f32a092-88a4-4c6b-a227-b4d79866a58f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:22:18.669680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:18.670780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:18.671824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 628/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:22:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:18.677992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195931c0-2aa6-48c0-9c5b-c8b37baa07e2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:18.782627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195931c0-2aa6-48c0-9c5b-c8b37baa07e2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:18.813008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:18.818202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:18.818815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:18.819099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:18.819374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:18.820042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:18.820383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:18.821746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:18.830384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 622/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:22:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:18.836930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32a092-88a4-4c6b-a227-b4d79866a58f None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:18.921937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:18.944822 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32a092-88a4-4c6b-a227-b4d79866a58f None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:18.977964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:18.978578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:18.978854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:18.979120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:18.989414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:18.990877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:19.004745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:19.005881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:19.007142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:19.007142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:19.007142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:19.007142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 629/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:19.011940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:19.118977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efe45f-0d57-4cde-bbd3-4f5963c79874 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:19.155104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:19.155772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:19.155982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:19.156316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:19.157015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:19.157350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:19.158720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:19.171826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 623/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:19.178043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:19.279698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a309b-ad7b-42d4-a969-4beaa986a123 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:19.306761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:19.314654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:19.315280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:19.316093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:19.316614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:19.323859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:19.325427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:22:19.337098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:19.338211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:19.339236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 630/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:19.344238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:19.401176 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:19.445133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca857bbe-056a-432d-9814-14862295ea41 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:19.477184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:19.477778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:19.478047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:19.478312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:19.479028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:19.479329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:19.480772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:19.497789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 624/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 13:22:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:19.503698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:19.606837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9956b-fe13-43fb-8abf-9f1f1e2dff98 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:19.641731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab5508-8d97-4e6c-b512-230222b23b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:19.642332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab5508-8d97-4e6c-b512-230222b23b20 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:19.642615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab5508-8d97-4e6c-b512-230222b23b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:19.642889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab5508-8d97-4e6c-b512-230222b23b20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:19.699935 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:19.712411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab5508-8d97-4e6c-b512-230222b23b20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:19.713854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab5508-8d97-4e6c-b512-230222b23b20 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:19.814574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 631/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:22:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:22:19.821001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:19.922615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c45d6-0ae0-4508-aeb9-285d7160a9bf None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:19.937008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:19.958581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:19.959145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:19.959429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:19.959707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:19.969482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:19.970953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:19.988440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:19.989531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:19.990438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:19.990438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:19.990438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:19.990438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 625/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:19.995317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab5508-8d97-4e6c-b512-230222b23b20 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:20.082812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:20.094969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab5508-8d97-4e6c-b512-230222b23b20 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:20.130048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:20.130645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:20.131246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:20.131519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:20.132154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:20.132493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:20.133849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:20.145213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 632/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:20.151110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:20.252722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d1a70-c6cf-4093-b26e-c2e45832c9e8 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:20.286871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:20.287487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:20.287768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:20.288040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:20.294850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:20.296324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:22:20.308229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:20.309290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:20.310283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 626/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:22:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:20.315459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:20.417507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc701f-d253-4c03-8871-e328cf3d1d35 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:20.454423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749d9c0-d056-4178-96da-853771ba36ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:20.455017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d9c0-d056-4178-96da-853771ba36ab None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:20.455296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d9c0-d056-4178-96da-853771ba36ab None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:20.455568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749d9c0-d056-4178-96da-853771ba36ab None admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:20.457034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749d9c0-d056-4178-96da-853771ba36ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:20.464329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749d9c0-d056-4178-96da-853771ba36ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:20.464329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749d9c0-d056-4178-96da-853771ba36ab None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:20.470778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 633/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:22:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:20.478166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:20.504852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:20.578685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db6910-035e-4a7b-9eda-26ec12133972 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:20.624768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa97b6-070f-42ef-a7bb-b3996b680db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:20.625359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa97b6-070f-42ef-a7bb-b3996b680db7 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:20.625640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa97b6-070f-42ef-a7bb-b3996b680db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:20.625912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa97b6-070f-42ef-a7bb-b3996b680db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:20.626630 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa97b6-070f-42ef-a7bb-b3996b680db7 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=ea29f79ed2af4341ab6f5ba50a47d8bf) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:20.627073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa97b6-070f-42ef-a7bb-b3996b680db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:20.629000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa97b6-070f-42ef-a7bb-b3996b680db7 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:20.634555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 627/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:22:20 2022] GET /identity/v3/role_assignments?user.id=ea29f79ed2af4341ab6f5ba50a47d8bf&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:22.509535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f4d70-ddf4-4abe-8bee-d77935885d07 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:22.510106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f4d70-ddf4-4abe-8bee-d77935885d07 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:22.510106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f4d70-ddf4-4abe-8bee-d77935885d07 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:22.511452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 634/1261] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:22.516815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b8230-f62f-498d-b441-3ce6c73abad6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:22.517292 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b8230-f62f-498d-b441-3ce6c73abad6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:22.517392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b8230-f62f-498d-b441-3ce6c73abad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:22.538483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-318b8230-f62f-498d-b441-3ce6c73abad6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:22.539732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-318b8230-f62f-498d-b441-3ce6c73abad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:22.611867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 628/1262] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:22.884968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f4d70-ddf4-4abe-8bee-d77935885d07 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:22.897096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:22.989360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f4d70-ddf4-4abe-8bee-d77935885d07 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:23.024859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97775631-453b-41a0-9cda-cee9ba0f3d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:23.025536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97775631-453b-41a0-9cda-cee9ba0f3d98 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:23.025817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97775631-453b-41a0-9cda-cee9ba0f3d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:23.026089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97775631-453b-41a0-9cda-cee9ba0f3d98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:23.036483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97775631-453b-41a0-9cda-cee9ba0f3d98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-97775631-453b-41a0-9cda-cee9ba0f3d98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:22:23.041274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:23.042387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:23.043439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 635/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:22:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:23.048792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b8230-f62f-498d-b441-3ce6c73abad6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:23.067910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:23.150546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b8230-f62f-498d-b441-3ce6c73abad6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:23.184295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:23.191621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:23.191621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:23.191621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:23.191621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:23.191621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:23.191621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:23.196374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 629/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:22:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:23.202414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97775631-453b-41a0-9cda-cee9ba0f3d98 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:23.270317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:23.304423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97775631-453b-41a0-9cda-cee9ba0f3d98 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:23.339433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:23.340055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:23.340367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:23.340643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:23.350710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:23.352152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:23.367266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:23.368476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:23.369408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:23.369408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:23.369408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:23.369408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 636/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:23.374718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:23.460846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:23.479083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfecf1-3650-4cbf-b009-106908049c9a None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:23.513290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:23.513896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:23.514172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:23.514445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:23.515150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:23.515457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:23.516844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:23.528452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 630/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:22:23.534803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:23.638486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14002e-5710-427d-b57e-e483d5644c6a None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:23.657598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:23.671810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:23.672454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:23.672732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:23.673000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:23.681076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:23.682499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:22:23.693439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:23.694585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:23.695953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 637/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:23.700398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:23.806519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c2886-6ab3-4319-a3bd-ac0da4a217ca None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:23.840079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:23.840700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:23.840977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:23.841244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:23.841958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:23.842260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:23.843670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:23.853441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:23.865982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 631/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 13:22:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:23.871584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:23.981305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927f26f-cd62-448f-a252-78eb3d603a73 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:24.016715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:24.017268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:24.017549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:24.017822 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:24.029140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:24.030578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:24.045136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:24.046349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:24.047706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:24.047706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:24.047706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:24.047706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 638/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:24.052616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:24.151671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d2d98-a4ba-47bf-a642-bded5f8874c1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:24.185356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:24.185950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:24.186226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:24.186495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:24.187127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:24.187425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:24.188843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:24.202437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 632/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:24.209130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:24.243722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:24.312691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3939988-ee7b-45e2-9e24-9adbbd137077 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:24.348974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:24.349596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:24.349877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:24.350148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:24.356847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:24.358274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:22:24.369893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:24.371019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:24.371998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 639/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:24.377393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:24.439524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:24.487042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe4c52-3720-44e3-ba57-6906d5944e7c None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:24.523795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bdf07-a9dc-4233-a409-b93e60643c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:24.524899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bdf07-a9dc-4233-a409-b93e60643c39 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:24.525389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bdf07-a9dc-4233-a409-b93e60643c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:24.525869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bdf07-a9dc-4233-a409-b93e60643c39 None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:24.527130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bdf07-a9dc-4233-a409-b93e60643c39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:24.527652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bdf07-a9dc-4233-a409-b93e60643c39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:24.530260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bdf07-a9dc-4233-a409-b93e60643c39 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:24.544094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 633/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:24.550587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:24.646102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:24.654723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd0ce4-d25b-4b13-8d04-2c1e6acf6c46 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:24.689880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dab00-8060-4f19-b08b-28abeac18a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:24.691193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dab00-8060-4f19-b08b-28abeac18a68 None admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:24.691475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dab00-8060-4f19-b08b-28abeac18a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:24.691744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dab00-8060-4f19-b08b-28abeac18a68 None admin] PATH_INFO: `/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/ea29f79ed2af4341ab6f5ba50a47d8bf/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:24.693676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dab00-8060-4f19-b08b-28abeac18a68 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2c71d386314c7190d1f2e2e3038636, user_id=ea29f79ed2af4341ab6f5ba50a47d8bf, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:24.695092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dab00-8060-4f19-b08b-28abeac18a68 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:24.777572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 640/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 13:22:24 2022] PUT /identity/v3/projects/2c2c71d386314c7190d1f2e2e3038636/users/ea29f79ed2af4341ab6f5ba50a47d8bf/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:22:26.680940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a469aef-64bf-4061-a01b-3fc7f403f661 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:26.681494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a469aef-64bf-4061-a01b-3fc7f403f661 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:26.681588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a469aef-64bf-4061-a01b-3fc7f403f661 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:26.682874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 634/1274] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:26.688204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2d9fb-0448-4c18-aa10-7d9f576be556 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:26.688615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2d9fb-0448-4c18-aa10-7d9f576be556 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:26.688762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2d9fb-0448-4c18-aa10-7d9f576be556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:26.708746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-24c2d9fb-0448-4c18-aa10-7d9f576be556 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:26.712320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-24c2d9fb-0448-4c18-aa10-7d9f576be556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:26.776481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 641/1275] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 13:22:27.051993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a469aef-64bf-4061-a01b-3fc7f403f661 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:27.153590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a469aef-64bf-4061-a01b-3fc7f403f661 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:27.171055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:27.188489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a433cca-a097-492f-be00-255bd08444bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:27.189117 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a433cca-a097-492f-be00-255bd08444bb None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:27.189397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a433cca-a097-492f-be00-255bd08444bb None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:27.189670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a433cca-a097-492f-be00-255bd08444bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:27.201664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a433cca-a097-492f-be00-255bd08444bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-9a433cca-a097-492f-be00-255bd08444bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 13:22:27.206751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:27.207864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:27.208945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 635/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 13:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:27.215036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2d9fb-0448-4c18-aa10-7d9f576be556 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:27.299336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:27.318128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2d9fb-0448-4c18-aa10-7d9f576be556 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:27.353844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:27.354440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:27.354708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:27.354973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:27.355633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:27.355930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:27.357316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:27.366359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 642/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 13:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:27.372289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a433cca-a097-492f-be00-255bd08444bb None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:27.478770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a433cca-a097-492f-be00-255bd08444bb None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:27.512953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:27.513572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:27.513848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:27.514115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:27.525108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:27.527337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:27.548412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:27.553453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:27.555314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:27.555314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:27.555314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:27.555314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 636/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:27.556909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:27.673283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba0479-eddc-4e4d-ae55-6ee2bccd19ef None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:27.708764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:27.709412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:27.709630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:27.709899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:27.710536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:27.710838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:27.712250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:27.724350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 643/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:27.730194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:27.762222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:27.834658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c2e04-50f4-48dd-a0fe-226742807ace None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:27.869617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:27.870241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:27.870524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:27.870799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:27.879334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:27.880801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 13:22:27.892366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:27.895191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:27.897308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 637/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:27.900689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:27.904920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:28.006353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e7578-4df0-4826-8fd6-ca1c6c9db133 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:28.040174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:28.040893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:28.041176 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:28.041448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:28.042167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:28.042469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:28.043888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:28.063448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 644/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 13:22:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:28.069708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:28.165475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:28.176167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5554eea-c292-4702-834c-a12795e69413 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:28.209198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870e446-94a4-45cb-adcc-897e390d18b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:28.209777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e446-94a4-45cb-adcc-897e390d18b2 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:28.210059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e446-94a4-45cb-adcc-897e390d18b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:28.210336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e446-94a4-45cb-adcc-897e390d18b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:28.279280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870e446-94a4-45cb-adcc-897e390d18b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:28.281263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870e446-94a4-45cb-adcc-897e390d18b2 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:28.371407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:28.382572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 638/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 13:22:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 13:22:28.388548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:28.453142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:28.494437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2999d9-ca7a-4a7b-b2b6-d1848ffca219 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:28.529207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:28.529880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:28.530111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:28.530420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:28.542785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:28.544373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 13:22:28.558641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:28.559768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 13:22:28.560792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 13:22:28.560792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 13:22:28.560792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:28.560792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 645/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:28.565734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870e446-94a4-45cb-adcc-897e390d18b2 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:28.676841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870e446-94a4-45cb-adcc-897e390d18b2 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:28.712138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:28.712762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:28.713043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:28.713322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:28.713954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:28.714256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:28.715618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:28.727605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 639/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 13:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:28.733385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:28.834548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db82a4-8615-4d45-8144-0a0d4e708989 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:28.849143 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:28.868590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:28.869205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:28.869486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:28.869754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:28.876632 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:28.878793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 13:22:28.889837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:28.890921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: ERROR keystone.server.flask.application Aug 24 13:22:28.891880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 646/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 13:22:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:28.897753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:28.950124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:28.999010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a6fd6-f770-4061-ba9b-0b3a2366ec75 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:29.032747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f11b2-3e55-4fb1-ad30-c8bdab98ca5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:29.033357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f11b2-3e55-4fb1-ad30-c8bdab98ca5f None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:29.033684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f11b2-3e55-4fb1-ad30-c8bdab98ca5f None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:29.033963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f11b2-3e55-4fb1-ad30-c8bdab98ca5f None admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:29.034808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f11b2-3e55-4fb1-ad30-c8bdab98ca5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:29.035110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f11b2-3e55-4fb1-ad30-c8bdab98ca5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:29.036570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f11b2-3e55-4fb1-ad30-c8bdab98ca5f None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:29.048860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 640/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 13:22:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:29.055364 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:29.162389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fc586-b695-4565-86d8-469da3ada734 None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:29.209538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0096b-a43c-428d-8b3b-ee7d20f51935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:29.210615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0096b-a43c-428d-8b3b-ee7d20f51935 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:29.211101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0096b-a43c-428d-8b3b-ee7d20f51935 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:29.211601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0096b-a43c-428d-8b3b-ee7d20f51935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:29.212619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0096b-a43c-428d-8b3b-ee7d20f51935 None admin] RBAC: Adding query filter params (role.id=b1b101411f4d4453a0eb2b378eeab684, scope.project.id=2c2c71d386314c7190d1f2e2e3038636, user.id=ea29f79ed2af4341ab6f5ba50a47d8bf) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:29.212938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0096b-a43c-428d-8b3b-ee7d20f51935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:29.214391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0096b-a43c-428d-8b3b-ee7d20f51935 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:29.225123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 647/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 13:22:29 2022] GET /identity/v3/role_assignments?user.id=ea29f79ed2af4341ab6f5ba50a47d8bf&scope.project.id=2c2c71d386314c7190d1f2e2e3038636&role.id=b1b101411f4d4453a0eb2b378eeab684 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:31.201020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e456311-a3b2-4823-bd67-4aac0c4af566 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:31.201700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e456311-a3b2-4823-bd67-4aac0c4af566 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:31.201700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e456311-a3b2-4823-bd67-4aac0c4af566 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:31.202893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 641/1288] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:31.208828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545ddc-7617-4403-b2ac-20eeb8cca709 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:31.209273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545ddc-7617-4403-b2ac-20eeb8cca709 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:31.209464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545ddc-7617-4403-b2ac-20eeb8cca709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:31.227943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-db545ddc-7617-4403-b2ac-20eeb8cca709 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:31.229181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-db545ddc-7617-4403-b2ac-20eeb8cca709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:31.258189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:31.294395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 648/1289] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:31.564078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e456311-a3b2-4823-bd67-4aac0c4af566 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:31.667807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e456311-a3b2-4823-bd67-4aac0c4af566 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:31.679072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:31.702813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:31.703422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:31.703698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:31.703965 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:31.704567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:31.705924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:31.714429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:31.716461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:31.717480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 642/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:22:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:31.721834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545ddc-7617-4403-b2ac-20eeb8cca709 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:31.822089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545ddc-7617-4403-b2ac-20eeb8cca709 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:31.855468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219e06-cad9-427f-922f-4b1b30819203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:31.856078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219e06-cad9-427f-922f-4b1b30819203 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:31.856366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219e06-cad9-427f-922f-4b1b30819203 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:31.856634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219e06-cad9-427f-922f-4b1b30819203 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:31.857249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13219e06-cad9-427f-922f-4b1b30819203 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:31.857546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13219e06-cad9-427f-922f-4b1b30819203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:31.858891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13219e06-cad9-427f-922f-4b1b30819203 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:31.864147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 649/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 13:22:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:31.869873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:31.971542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57a5e9-bf28-4566-a600-609af71179bd None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:32.007037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0d45c-1852-4ae2-9984-ca2dbd32f01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:32.007640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0d45c-1852-4ae2-9984-ca2dbd32f01d None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:32.007919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0d45c-1852-4ae2-9984-ca2dbd32f01d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:32.008215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0d45c-1852-4ae2-9984-ca2dbd32f01d None admin] PATH_INFO: `/v3/services` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:32.008856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0d45c-1852-4ae2-9984-ca2dbd32f01d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:32.009161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0d45c-1852-4ae2-9984-ca2dbd32f01d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:32.010662 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0d45c-1852-4ae2-9984-ca2dbd32f01d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:32.015383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 643/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 13:22:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:34.113260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ab99c-e285-4ec7-b416-f83f88485cca None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:34.113848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ab99c-e285-4ec7-b416-f83f88485cca None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:34.113848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ab99c-e285-4ec7-b416-f83f88485cca None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:34.115114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 650/1293] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:34.120674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c857ebd-6411-4ae9-8f15-30b6466c4abd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:34.121174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c857ebd-6411-4ae9-8f15-30b6466c4abd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:34.121284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c857ebd-6411-4ae9-8f15-30b6466c4abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:34.139342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6c857ebd-6411-4ae9-8f15-30b6466c4abd None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:34.140564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6c857ebd-6411-4ae9-8f15-30b6466c4abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:34.170692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:34.210465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 644/1294] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:34.481680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ab99c-e285-4ec7-b416-f83f88485cca None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:34.514993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:34.592081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ab99c-e285-4ec7-b416-f83f88485cca None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:34.626392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03754d-24bb-4789-9425-986c9f6917f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:34.626989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03754d-24bb-4789-9425-986c9f6917f6 None admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:34.627360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03754d-24bb-4789-9425-986c9f6917f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:34.627637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03754d-24bb-4789-9425-986c9f6917f6 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:34.628899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03754d-24bb-4789-9425-986c9f6917f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:34.636422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03754d-24bb-4789-9425-986c9f6917f6 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:34.739259 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 651/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 13:22:34 2022] POST /identity/v3/services => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:22:36.630574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3768ead-5e65-4e39-93f7-9f851817f263 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:36.631154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3768ead-5e65-4e39-93f7-9f851817f263 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:36.631254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3768ead-5e65-4e39-93f7-9f851817f263 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:36.632610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 645/1296] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:36.638746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852ad98-ee11-41bc-9f04-10d46cda6b84 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:36.639319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852ad98-ee11-41bc-9f04-10d46cda6b84 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:36.639319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852ad98-ee11-41bc-9f04-10d46cda6b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:36.661515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3852ad98-ee11-41bc-9f04-10d46cda6b84 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:36.662790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3852ad98-ee11-41bc-9f04-10d46cda6b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:36.749388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 652/1297] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:37.019436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3768ead-5e65-4e39-93f7-9f851817f263 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:37.123251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3768ead-5e65-4e39-93f7-9f851817f263 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:37.158096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:37.158714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:37.159049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:37.159383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:37.159963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:37.161418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:37.169948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:37.171352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:37.172739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 646/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:22:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:37.177634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852ad98-ee11-41bc-9f04-10d46cda6b84 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:37.196325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:37.277566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852ad98-ee11-41bc-9f04-10d46cda6b84 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:37.311692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:37.312328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:37.312607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:37.312872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:37.313537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:37.313799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:37.315684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:37.320760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 653/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 13:22:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:37.326939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:37.358490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:37.429634 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67e677-32a8-4d3b-bf3b-9ee59d245ed1 None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:37.463054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dd507-a494-48e0-b8c6-2af2dc47f7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:37.463651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd507-a494-48e0-b8c6-2af2dc47f7ee None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:37.463932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd507-a494-48e0-b8c6-2af2dc47f7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:37.464227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dd507-a494-48e0-b8c6-2af2dc47f7ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:37.464924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dd507-a494-48e0-b8c6-2af2dc47f7ee None admin] RBAC: Adding query filter params (interface=public, service_id=a10658e514ec42d39b3201e03a17a0eb, region_id=RegionOne) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:37.465231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dd507-a494-48e0-b8c6-2af2dc47f7ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:37.466583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dd507-a494-48e0-b8c6-2af2dc47f7ee None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:37.471493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 647/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 13:22:37 2022] GET /identity/v3/endpoints?service_id=a10658e514ec42d39b3201e03a17a0eb&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:37.477821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:37.561117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:37.579750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f145b-9a3b-43fd-bca8-b1abeb9a560c None admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:37.613682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3abb8e-6afc-41f3-aea6-8e7260ba1ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:37.614274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3abb8e-6afc-41f3-aea6-8e7260ba1ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:37.614554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3abb8e-6afc-41f3-aea6-8e7260ba1ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:37.614823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3abb8e-6afc-41f3-aea6-8e7260ba1ee3 None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:37.615425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3abb8e-6afc-41f3-aea6-8e7260ba1ee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:37.617092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3abb8e-6afc-41f3-aea6-8e7260ba1ee3 None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:37.623838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 654/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 13:22:37 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:22:39.539429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfba2f-4531-4f8d-a638-2d3f9f245eb4 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:39.540004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfba2f-4531-4f8d-a638-2d3f9f245eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:39.540004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfba2f-4531-4f8d-a638-2d3f9f245eb4 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:39.541354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 648/1302] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:22:39.546820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35693fc-46b5-41be-bb52-74c293130d8a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:39.547226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35693fc-46b5-41be-bb52-74c293130d8a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:39.547380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35693fc-46b5-41be-bb52-74c293130d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:39.566114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c35693fc-46b5-41be-bb52-74c293130d8a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:22:39.567292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c35693fc-46b5-41be-bb52-74c293130d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:22:39.650153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 655/1303] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:22:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 13:22:39.917914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfba2f-4531-4f8d-a638-2d3f9f245eb4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:40.016388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:40.023561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfba2f-4531-4f8d-a638-2d3f9f245eb4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:40.056778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:40.057393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:40.057670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:40.057940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:40.058515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:40.059859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 13:22:40.068687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 13:22:40.069913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application with Lock( Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self._enter() Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application return self.creator() Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application created_value = creator( Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: ERROR keystone.server.flask.application Aug 24 13:22:40.070945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 649/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 13:22:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 13:22:40.075746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35693fc-46b5-41be-bb52-74c293130d8a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:40.153845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:40.178694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35693fc-46b5-41be-bb52-74c293130d8a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:40.212842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865813b9-fc49-4773-b412-b1a07b6b3cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:40.213440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865813b9-fc49-4773-b412-b1a07b6b3cca None admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:40.213722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865813b9-fc49-4773-b412-b1a07b6b3cca None admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:40.213997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865813b9-fc49-4773-b412-b1a07b6b3cca None admin] PATH_INFO: `/v3/services` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:40.214625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865813b9-fc49-4773-b412-b1a07b6b3cca None admin] RBAC: Adding query filter params (name=placement) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:22:40.214930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865813b9-fc49-4773-b412-b1a07b6b3cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:40.216465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865813b9-fc49-4773-b412-b1a07b6b3cca None admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:40.222014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 656/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 13:22:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:22:40.227876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:22:40.331695 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902ae4-8d39-45d7-bc4a-ae198d2dcedc None admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:22:40.365632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f8b03-0e24-4535-bb90-8920044ae69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:22:40.366223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f8b03-0e24-4535-bb90-8920044ae69d None admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:22:40.366499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f8b03-0e24-4535-bb90-8920044ae69d None admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:22:40.366767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f8b03-0e24-4535-bb90-8920044ae69d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:22:40.367467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f8b03-0e24-4535-bb90-8920044ae69d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:22:40.368894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f8b03-0e24-4535-bb90-8920044ae69d None admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:22:40.464561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 650/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:22:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:22:50.245789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=30 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:22:50.459559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:53.121069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593aa3b4-7f3d-427e-aa48-ba70a14b140b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:53.121644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593aa3b4-7f3d-427e-aa48-ba70a14b140b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:53.121644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593aa3b4-7f3d-427e-aa48-ba70a14b140b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:53.122952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 657/1307] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:29:53.128565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f0af0-0fe3-4ec1-8768-dfefa233aad4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:53.129031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f0af0-0fe3-4ec1-8768-dfefa233aad4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:53.129115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f0af0-0fe3-4ec1-8768-dfefa233aad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:53.209049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5f5f0af0-0fe3-4ec1-8768-dfefa233aad4 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:53.210283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5f5f0af0-0fe3-4ec1-8768-dfefa233aad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:53.349804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 651/1308] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:29:53.640837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a74695-e9fb-4409-a5fe-2f22f3aecc71 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:53.641207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a74695-e9fb-4409-a5fe-2f22f3aecc71 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:53.641592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a74695-e9fb-4409-a5fe-2f22f3aecc71 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:53.643042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 658/1309] 10.222.0.73 () {54 vars in 965 bytes} [Wed Aug 24 13:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:29:53.651296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9b3c2-2913-45be-b2c9-7021ce26695d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:53.651793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9b3c2-2913-45be-b2c9-7021ce26695d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:53.652327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9b3c2-2913-45be-b2c9-7021ce26695d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:53.746537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8be9b3c2-2913-45be-b2c9-7021ce26695d None None] MFA Rules not processed for user `18ca375f53414475b75dae308cd19fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:53.746537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8be9b3c2-2913-45be-b2c9-7021ce26695d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:53.774423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:53.851006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 652/1310] 10.222.0.73 () {58 vars in 1062 bytes} [Wed Aug 24 13:29:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:29:53.860525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a74695-e9fb-4409-a5fe-2f22f3aecc71 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:53.862604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-15a74695-e9fb-4409-a5fe-2f22f3aecc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:54.014967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a74695-e9fb-4409-a5fe-2f22f3aecc71 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:54.109652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf91c8f-2b5b-4f16-9ab0-38d9203e8192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:54.110441 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91c8f-2b5b-4f16-9ab0-38d9203e8192 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:54.110915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91c8f-2b5b-4f16-9ab0-38d9203e8192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:54.111298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91c8f-2b5b-4f16-9ab0-38d9203e8192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:54.115472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dcf91c8f-2b5b-4f16-9ab0-38d9203e8192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:54.206704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:54.280749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf91c8f-2b5b-4f16-9ab0-38d9203e8192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:54.282509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf91c8f-2b5b-4f16-9ab0-38d9203e8192 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:54.306417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 659/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:29:56.669199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5900fe-88f4-4612-a9d7-2dbf448bac11 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:56.669764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5900fe-88f4-4612-a9d7-2dbf448bac11 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:56.669764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5900fe-88f4-4612-a9d7-2dbf448bac11 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:56.671630 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 653/1312] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:29:56.677448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e894cd-b4ce-4dc2-9289-6c35cdac24bf None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:56.677940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e894cd-b4ce-4dc2-9289-6c35cdac24bf None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:56.678059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e894cd-b4ce-4dc2-9289-6c35cdac24bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:56.768302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a5e894cd-b4ce-4dc2-9289-6c35cdac24bf None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:29:56.769532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a5e894cd-b4ce-4dc2-9289-6c35cdac24bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:56.878409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 660/1313] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:29:57.164470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5900fe-88f4-4612-a9d7-2dbf448bac11 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:29:57.166021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3d5900fe-88f4-4612-a9d7-2dbf448bac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:57.278920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5900fe-88f4-4612-a9d7-2dbf448bac11 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:29:57.358207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:29:57.360957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf532cb-f88c-485b-9a0b-1448a1f1260d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:29:57.361566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf532cb-f88c-485b-9a0b-1448a1f1260d service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:29:57.361850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf532cb-f88c-485b-9a0b-1448a1f1260d service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:29:57.362130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf532cb-f88c-485b-9a0b-1448a1f1260d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:29:57.363759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ddf532cb-f88c-485b-9a0b-1448a1f1260d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:29:57.489114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf532cb-f88c-485b-9a0b-1448a1f1260d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:29:57.490480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf532cb-f88c-485b-9a0b-1448a1f1260d service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:29:57.508885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 654/1314] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:00.142871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654630e8-0624-44f5-94b4-abc9ab45fe5b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:00.143405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654630e8-0624-44f5-94b4-abc9ab45fe5b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:00.143405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654630e8-0624-44f5-94b4-abc9ab45fe5b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:00.144814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 661/1315] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:00.150139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a98b3d-5343-486b-bebf-2b001250272e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:00.150577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a98b3d-5343-486b-bebf-2b001250272e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:00.150672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a98b3d-5343-486b-bebf-2b001250272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:00.218381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-42a98b3d-5343-486b-bebf-2b001250272e None None] MFA Rules not processed for user `07197b17f7014c6192df2c1a77382f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:00.219621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-42a98b3d-5343-486b-bebf-2b001250272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:00.272854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:00.317827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 655/1316] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:00 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:00.634331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25b9c5-d333-4420-8556-7ac9c026bb17 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:00.634644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25b9c5-d333-4420-8556-7ac9c026bb17 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:00.634925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25b9c5-d333-4420-8556-7ac9c026bb17 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:00.636346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 662/1317] 10.222.0.73 () {54 vars in 965 bytes} [Wed Aug 24 13:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:00.643769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71775ac3-2ea1-4b09-94f6-c5cbebef4f53 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:00.644052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71775ac3-2ea1-4b09-94f6-c5cbebef4f53 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:00.644377 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71775ac3-2ea1-4b09-94f6-c5cbebef4f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:00.731516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-71775ac3-2ea1-4b09-94f6-c5cbebef4f53 None None] MFA Rules not processed for user `18ca375f53414475b75dae308cd19fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:00.732848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-71775ac3-2ea1-4b09-94f6-c5cbebef4f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:00.837512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 656/1318] 10.222.0.73 () {58 vars in 1062 bytes} [Wed Aug 24 13:30:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:30:00.847250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25b9c5-d333-4420-8556-7ac9c026bb17 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:00.848824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7f25b9c5-d333-4420-8556-7ac9c026bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:00.858753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:00.962504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25b9c5-d333-4420-8556-7ac9c026bb17 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:01.037631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42debf87-3ca3-4763-9b46-5c88d4ea7336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:01.038247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42debf87-3ca3-4763-9b46-5c88d4ea7336 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:01.038530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42debf87-3ca3-4763-9b46-5c88d4ea7336 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:01.038806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42debf87-3ca3-4763-9b46-5c88d4ea7336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:01.040464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-42debf87-3ca3-4763-9b46-5c88d4ea7336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:01.104581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:01.152816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42debf87-3ca3-4763-9b46-5c88d4ea7336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:01.154269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42debf87-3ca3-4763-9b46-5c88d4ea7336 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:01.173415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 663/1319] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:04.337017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a0ff-a1a3-43e3-a6b5-05640ea123ba None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:04.337594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a0ff-a1a3-43e3-a6b5-05640ea123ba None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:04.337594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a0ff-a1a3-43e3-a6b5-05640ea123ba None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:04.338910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 657/1320] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:04.344887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46861ecc-add6-4011-a61e-6e8ce56ddeeb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:04.345329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46861ecc-add6-4011-a61e-6e8ce56ddeeb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:04.345446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46861ecc-add6-4011-a61e-6e8ce56ddeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:04.414469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-46861ecc-add6-4011-a61e-6e8ce56ddeeb None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:04.415716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-46861ecc-add6-4011-a61e-6e8ce56ddeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:04.520202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 664/1321] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:04.810058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8a0ff-a1a3-43e3-a6b5-05640ea123ba None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:04.811763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0bb8a0ff-a1a3-43e3-a6b5-05640ea123ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:04.867298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:04.924119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8a0ff-a1a3-43e3-a6b5-05640ea123ba None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:05.006088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca13ad-e70c-4f09-85f7-3698ccb7c9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:05.006648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca13ad-e70c-4f09-85f7-3698ccb7c9ef service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:05.006931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca13ad-e70c-4f09-85f7-3698ccb7c9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:05.007254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca13ad-e70c-4f09-85f7-3698ccb7c9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:05.009252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f5ca13ad-e70c-4f09-85f7-3698ccb7c9ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:05.114167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:05.127341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca13ad-e70c-4f09-85f7-3698ccb7c9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:05.128902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca13ad-e70c-4f09-85f7-3698ccb7c9ef service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:05.149594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 658/1322] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:08.284096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde21e0-979e-4f2d-b249-b5ce457cf52d None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:08.284697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde21e0-979e-4f2d-b249-b5ce457cf52d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:08.284888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde21e0-979e-4f2d-b249-b5ce457cf52d None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:08.286302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 665/1323] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:08.292009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92a5d9-b93b-46af-86c5-7ecb975271f2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:08.292555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92a5d9-b93b-46af-86c5-7ecb975271f2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:08.292662 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92a5d9-b93b-46af-86c5-7ecb975271f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:08.369179 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ae92a5d9-b93b-46af-86c5-7ecb975271f2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:08.370314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ae92a5d9-b93b-46af-86c5-7ecb975271f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:08.484733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 659/1324] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:08.772568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde21e0-979e-4f2d-b249-b5ce457cf52d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:08.774171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9fde21e0-979e-4f2d-b249-b5ce457cf52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:08.800054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:08.909584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde21e0-979e-4f2d-b249-b5ce457cf52d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:08.990759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555961e-72f0-46fd-869b-b1330515cb60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:08.991371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555961e-72f0-46fd-869b-b1330515cb60 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:08.991652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555961e-72f0-46fd-869b-b1330515cb60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:08.991928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555961e-72f0-46fd-869b-b1330515cb60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:08.993621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6555961e-72f0-46fd-869b-b1330515cb60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:09.068123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:09.107998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555961e-72f0-46fd-869b-b1330515cb60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:09.110377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555961e-72f0-46fd-869b-b1330515cb60 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:09.131000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 666/1325] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:11.191777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72d46-8e66-4b1f-9dad-14449f1a06e3 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:11.192327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72d46-8e66-4b1f-9dad-14449f1a06e3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:11.192784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba72d46-8e66-4b1f-9dad-14449f1a06e3 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:11.194975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 660/1326] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:11.200681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c7cf-9213-4572-a385-059e0713672a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:11.201286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c7cf-9213-4572-a385-059e0713672a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:11.201286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0c7cf-9213-4572-a385-059e0713672a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:11.292579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bea0c7cf-9213-4572-a385-059e0713672a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:11.294091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bea0c7cf-9213-4572-a385-059e0713672a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:11.409355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 667/1327] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:11.696819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba72d46-8e66-4b1f-9dad-14449f1a06e3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:11.698870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-eba72d46-8e66-4b1f-9dad-14449f1a06e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:11.770295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:11.822340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba72d46-8e66-4b1f-9dad-14449f1a06e3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:11.960304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b4a52-3508-4848-bb5d-0214af47b0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:11.961071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b4a52-3508-4848-bb5d-0214af47b0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:11.961530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b4a52-3508-4848-bb5d-0214af47b0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:11.961886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b4a52-3508-4848-bb5d-0214af47b0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:11.963506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ac6b4a52-3508-4848-bb5d-0214af47b0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:12.115195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:12.116882 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b4a52-3508-4848-bb5d-0214af47b0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:12.118342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b4a52-3508-4848-bb5d-0214af47b0b5 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:12.139180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 661/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:14.176592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95144945-56ad-4529-b713-dc4ecbe0a660 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:14.177183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95144945-56ad-4529-b713-dc4ecbe0a660 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:14.177183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95144945-56ad-4529-b713-dc4ecbe0a660 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:14.178901 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 668/1329] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:14.185290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98683a9-8660-4eb8-9962-d7be5905e5a2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:14.185866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98683a9-8660-4eb8-9962-d7be5905e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:14.185866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98683a9-8660-4eb8-9962-d7be5905e5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:14.266661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e98683a9-8660-4eb8-9962-d7be5905e5a2 None None] MFA Rules not processed for user `07197b17f7014c6192df2c1a77382f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:14.267881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e98683a9-8660-4eb8-9962-d7be5905e5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:14.371111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 662/1330] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:14.659080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95144945-56ad-4529-b713-dc4ecbe0a660 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:14.660743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-95144945-56ad-4529-b713-dc4ecbe0a660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:14.697157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:14.775962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95144945-56ad-4529-b713-dc4ecbe0a660 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:14.872073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847ebc-8544-49d1-bea7-ab4c5b488771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:14.872731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847ebc-8544-49d1-bea7-ab4c5b488771 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:14.873011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847ebc-8544-49d1-bea7-ab4c5b488771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:14.873284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847ebc-8544-49d1-bea7-ab4c5b488771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:14.875593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-99847ebc-8544-49d1-bea7-ab4c5b488771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:14.963862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:15.001906 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847ebc-8544-49d1-bea7-ab4c5b488771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:15.003476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847ebc-8544-49d1-bea7-ab4c5b488771 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:15.023586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 669/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:17.465921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5305a9b-769a-461b-a203-1807e578e0d3 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:17.466502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5305a9b-769a-461b-a203-1807e578e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:17.466502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5305a9b-769a-461b-a203-1807e578e0d3 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:17.467814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 663/1332] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:17.473494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6337-a89c-4f17-b5df-c05231581f73 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:17.473912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6337-a89c-4f17-b5df-c05231581f73 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:17.474046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a6337-a89c-4f17-b5df-c05231581f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:17.560888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-899a6337-a89c-4f17-b5df-c05231581f73 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:17.563551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-899a6337-a89c-4f17-b5df-c05231581f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:17.695889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 670/1333] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:17.987363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5305a9b-769a-461b-a203-1807e578e0d3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:17.988964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c5305a9b-769a-461b-a203-1807e578e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:18.064267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:18.099682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5305a9b-769a-461b-a203-1807e578e0d3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:18.178195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9af170b-cb32-446c-935f-ba9a66c8cd44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:18.178830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af170b-cb32-446c-935f-ba9a66c8cd44 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:18.179112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af170b-cb32-446c-935f-ba9a66c8cd44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:18.179386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af170b-cb32-446c-935f-ba9a66c8cd44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:18.181306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a9af170b-cb32-446c-935f-ba9a66c8cd44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:18.294475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af170b-cb32-446c-935f-ba9a66c8cd44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:18.295968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9af170b-cb32-446c-935f-ba9a66c8cd44 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:18.306526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:18.318681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 664/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:20.328330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98384c80-7839-4fb0-a7cc-a7d423ec9f63 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:20.328903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98384c80-7839-4fb0-a7cc-a7d423ec9f63 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:20.328903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98384c80-7839-4fb0-a7cc-a7d423ec9f63 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:20.330350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 671/1335] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:20.336141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a0b39-6cce-47a8-9f08-e8cb6748977a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:20.336699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a0b39-6cce-47a8-9f08-e8cb6748977a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:20.336804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a0b39-6cce-47a8-9f08-e8cb6748977a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:20.410519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1c0a0b39-6cce-47a8-9f08-e8cb6748977a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:20.411772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1c0a0b39-6cce-47a8-9f08-e8cb6748977a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:20.515054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 665/1336] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:20.928468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98384c80-7839-4fb0-a7cc-a7d423ec9f63 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:20.930343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-98384c80-7839-4fb0-a7cc-a7d423ec9f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:20.971834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:21.039237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98384c80-7839-4fb0-a7cc-a7d423ec9f63 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:21.115954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23435158-e948-4fb7-b216-fd737f26570b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:21.116583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23435158-e948-4fb7-b216-fd737f26570b service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:21.116860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23435158-e948-4fb7-b216-fd737f26570b service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:21.117131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23435158-e948-4fb7-b216-fd737f26570b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:21.118727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-23435158-e948-4fb7-b216-fd737f26570b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:21.204984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:21.240368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23435158-e948-4fb7-b216-fd737f26570b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:21.241812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23435158-e948-4fb7-b216-fd737f26570b service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:21.260991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 672/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:30:24.234073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea53f4-55ff-43a0-9f39-7a96853964af None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:24.234628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea53f4-55ff-43a0-9f39-7a96853964af None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:24.234628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea53f4-55ff-43a0-9f39-7a96853964af None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:24.235992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 666/1338] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:24.241671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee861b-b6b0-4be6-9628-6dfa9104c57f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:24.242131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee861b-b6b0-4be6-9628-6dfa9104c57f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:24.242253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee861b-b6b0-4be6-9628-6dfa9104c57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:24.325567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-07ee861b-b6b0-4be6-9628-6dfa9104c57f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:24.326843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-07ee861b-b6b0-4be6-9628-6dfa9104c57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:24.439369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 673/1339] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:24.750083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea53f4-55ff-43a0-9f39-7a96853964af None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:24.751623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-56ea53f4-55ff-43a0-9f39-7a96853964af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:24.840476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:24.869680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea53f4-55ff-43a0-9f39-7a96853964af None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:24.963582 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef40de-b2c3-439d-8e9e-b84d9c6a58cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:24.964226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef40de-b2c3-439d-8e9e-b84d9c6a58cf service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:24.964512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef40de-b2c3-439d-8e9e-b84d9c6a58cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:24.964782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef40de-b2c3-439d-8e9e-b84d9c6a58cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:24.966425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-06ef40de-b2c3-439d-8e9e-b84d9c6a58cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:25.098780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef40de-b2c3-439d-8e9e-b84d9c6a58cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:25.100261 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef40de-b2c3-439d-8e9e-b84d9c6a58cf service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:25.120238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 667/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:30:29.599479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1004eb1-7992-4049-be1c-8c538e85f009 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:29.600039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1004eb1-7992-4049-be1c-8c538e85f009 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:29.600039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1004eb1-7992-4049-be1c-8c538e85f009 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:29.603487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 674/1341] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:29.607209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0565cc0-b362-469b-90c0-95f1b319b3df None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:29.610153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0565cc0-b362-469b-90c0-95f1b319b3df None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:29.610153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0565cc0-b362-469b-90c0-95f1b319b3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:29.614212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:29.682743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b0565cc0-b362-469b-90c0-95f1b319b3df None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:29.683974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b0565cc0-b362-469b-90c0-95f1b319b3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:29.817959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 668/1342] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:30.113371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1004eb1-7992-4049-be1c-8c538e85f009 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:30.114999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e1004eb1-7992-4049-be1c-8c538e85f009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:30.170879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:30.236871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1004eb1-7992-4049-be1c-8c538e85f009 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:30.319509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0d9e3-e4e0-4e38-bd43-79d5f1cadfe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:30.320116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d9e3-e4e0-4e38-bd43-79d5f1cadfe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:30.320455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d9e3-e4e0-4e38-bd43-79d5f1cadfe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:30.320732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d9e3-e4e0-4e38-bd43-79d5f1cadfe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:30.322407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-51a0d9e3-e4e0-4e38-bd43-79d5f1cadfe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:30.459265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:30.479828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0d9e3-e4e0-4e38-bd43-79d5f1cadfe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:30.482042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0d9e3-e4e0-4e38-bd43-79d5f1cadfe8 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:30.504517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 675/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:33.688978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e187496-765c-4487-bf96-0fae5cd2708c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:33.689493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e187496-765c-4487-bf96-0fae5cd2708c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:33.689493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e187496-765c-4487-bf96-0fae5cd2708c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:33.690784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 669/1344] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:33.696651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb6b6-cc9a-413e-a40c-46cb01a87fcb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:33.697137 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb6b6-cc9a-413e-a40c-46cb01a87fcb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:33.697232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdb6b6-cc9a-413e-a40c-46cb01a87fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:33.775443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6bcdb6b6-cc9a-413e-a40c-46cb01a87fcb None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:33.776685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6bcdb6b6-cc9a-413e-a40c-46cb01a87fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:33.894479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 676/1345] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:34.183722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e187496-765c-4487-bf96-0fae5cd2708c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:34.186982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8e187496-765c-4487-bf96-0fae5cd2708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:34.301426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:34.329059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e187496-765c-4487-bf96-0fae5cd2708c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:34.415489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272de690-fb29-48c2-887c-9ed2e8cd5d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:34.416353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272de690-fb29-48c2-887c-9ed2e8cd5d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:34.416780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272de690-fb29-48c2-887c-9ed2e8cd5d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:34.417160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272de690-fb29-48c2-887c-9ed2e8cd5d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:34.419451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-272de690-fb29-48c2-887c-9ed2e8cd5d2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:34.550259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272de690-fb29-48c2-887c-9ed2e8cd5d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:34.551784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272de690-fb29-48c2-887c-9ed2e8cd5d2f service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:34.575062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 670/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:38.964678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199745b-54e7-46f2-a362-da2df08e44f3 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:38.965155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199745b-54e7-46f2-a362-da2df08e44f3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:38.965250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199745b-54e7-46f2-a362-da2df08e44f3 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:38.967308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 677/1347] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:38.978134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fbb40-2080-4605-8db4-563da48f2a19 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:38.978653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fbb40-2080-4605-8db4-563da48f2a19 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:38.978876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fbb40-2080-4605-8db4-563da48f2a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:39.003895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:39.083482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b10fbb40-2080-4605-8db4-563da48f2a19 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:39.084754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b10fbb40-2080-4605-8db4-563da48f2a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:39.229258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 671/1348] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:39.514599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199745b-54e7-46f2-a362-da2df08e44f3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:39.516311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f199745b-54e7-46f2-a362-da2df08e44f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:39.587874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:39.642220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199745b-54e7-46f2-a362-da2df08e44f3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:39.733336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695bf5b-9a90-4ce8-ad83-1636d14ba778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:39.734240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695bf5b-9a90-4ce8-ad83-1636d14ba778 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:39.734528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695bf5b-9a90-4ce8-ad83-1636d14ba778 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:39.734799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695bf5b-9a90-4ce8-ad83-1636d14ba778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:39.736497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7695bf5b-9a90-4ce8-ad83-1636d14ba778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:39.874167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:39.877818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695bf5b-9a90-4ce8-ad83-1636d14ba778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:39.879290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7695bf5b-9a90-4ce8-ad83-1636d14ba778 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:39.897887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 678/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:30:45.517846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590c808-c472-44b2-9e06-a0b7f0409f7a None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:45.518228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590c808-c472-44b2-9e06-a0b7f0409f7a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:45.518228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590c808-c472-44b2-9e06-a0b7f0409f7a None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:45.525152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 672/1350] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:30:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:45.529932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197a5ed-d08a-473d-b509-a4c35529a90f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:45.530421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197a5ed-d08a-473d-b509-a4c35529a90f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:45.530421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197a5ed-d08a-473d-b509-a4c35529a90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:45.653061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7197a5ed-d08a-473d-b509-a4c35529a90f None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:45.655635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7197a5ed-d08a-473d-b509-a4c35529a90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:45.881581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 679/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:30:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:49.234476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=59 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:51.428220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf030-c7e5-44cf-a44b-036cb203ef91 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:51.429175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf030-c7e5-44cf-a44b-036cb203ef91 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:51.429619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf030-c7e5-44cf-a44b-036cb203ef91 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:51.431536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 673/1352] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:30:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:51.437944 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a623-6087-44fc-9aeb-5713a5c0168a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:51.438389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a623-6087-44fc-9aeb-5713a5c0168a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:51.438620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554a623-6087-44fc-9aeb-5713a5c0168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:51.521669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:51.554126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e554a623-6087-44fc-9aeb-5713a5c0168a None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:51.555711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e554a623-6087-44fc-9aeb-5713a5c0168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:51.689589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 680/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:30:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:53.000854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515afd12-01c0-4779-8f32-37f91fcaf99b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:53.001371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515afd12-01c0-4779-8f32-37f91fcaf99b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:53.001371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515afd12-01c0-4779-8f32-37f91fcaf99b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:53.002932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 674/1354] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:53.008582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11975a-d224-43cf-9ff8-06e9460ffeb5 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:53.009148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11975a-d224-43cf-9ff8-06e9460ffeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:53.009244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11975a-d224-43cf-9ff8-06e9460ffeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:53.094324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-db11975a-d224-43cf-9ff8-06e9460ffeb5 None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:53.095546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-db11975a-d224-43cf-9ff8-06e9460ffeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:53.138214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:53.206204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 681/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:30:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:56.312663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b29b-3acf-4ef3-9f04-12891fae3ced None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:56.313228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b29b-3acf-4ef3-9f04-12891fae3ced None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:56.313228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916b29b-3acf-4ef3-9f04-12891fae3ced None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:56.314571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 675/1356] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:56.320123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d0a9-010f-4b21-9fba-41b1f9f672b9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:56.320573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d0a9-010f-4b21-9fba-41b1f9f672b9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:56.320897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98d0a9-010f-4b21-9fba-41b1f9f672b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:56.409483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bd98d0a9-010f-4b21-9fba-41b1f9f672b9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:56.411304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bd98d0a9-010f-4b21-9fba-41b1f9f672b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:56.524746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 682/1357] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:56.851897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90aa42-06df-4cf3-9e09-81cc66a07585 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:56.852696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90aa42-06df-4cf3-9e09-81cc66a07585 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:56.853207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90aa42-06df-4cf3-9e09-81cc66a07585 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:56.855419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 676/1358] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 13:30:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:56.865108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21084e-a814-438a-8198-f31daac52caa None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:56.865265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21084e-a814-438a-8198-f31daac52caa None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:56.869423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21084e-a814-438a-8198-f31daac52caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:56.882527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:56.956095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9b21084e-a814-438a-8198-f31daac52caa None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:56.957848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9b21084e-a814-438a-8198-f31daac52caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:57.095973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 683/1359] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:30:57.104775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90aa42-06df-4cf3-9e09-81cc66a07585 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:30:57.106329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ff90aa42-06df-4cf3-9e09-81cc66a07585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:57.275368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90aa42-06df-4cf3-9e09-81cc66a07585 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:30:57.353651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:57.395622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c8cdc-d31a-4548-8b68-9a427accfb2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:30:57.396264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c8cdc-d31a-4548-8b68-9a427accfb2f service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:57.396550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c8cdc-d31a-4548-8b68-9a427accfb2f service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:57.396824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c8cdc-d31a-4548-8b68-9a427accfb2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:57.398499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4c0c8cdc-d31a-4548-8b68-9a427accfb2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:57.501049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c8cdc-d31a-4548-8b68-9a427accfb2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:30:57.502606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c8cdc-d31a-4548-8b68-9a427accfb2f service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:30:57.522688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 677/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:30:59.701483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02890beb-6bd1-45d6-a6d0-5db715986c55 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:59.702007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02890beb-6bd1-45d6-a6d0-5db715986c55 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:59.702007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02890beb-6bd1-45d6-a6d0-5db715986c55 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:59.703419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 684/1361] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:30:59.708737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd5a6f-5e86-43ac-9a3c-ac8c6b682ba4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:30:59.709188 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd5a6f-5e86-43ac-9a3c-ac8c6b682ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:30:59.709278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd5a6f-5e86-43ac-9a3c-ac8c6b682ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:30:59.789712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3bdd5a6f-5e86-43ac-9a3c-ac8c6b682ba4 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:30:59.791051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3bdd5a6f-5e86-43ac-9a3c-ac8c6b682ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:30:59.837483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:30:59.907916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 678/1362] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:00.198158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02890beb-6bd1-45d6-a6d0-5db715986c55 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:00.199889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-02890beb-6bd1-45d6-a6d0-5db715986c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:00.287308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:00.386761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02890beb-6bd1-45d6-a6d0-5db715986c55 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:00.521472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d303b65-4378-44af-8d15-763d00ba4e56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:00.522237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d303b65-4378-44af-8d15-763d00ba4e56 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:00.522646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d303b65-4378-44af-8d15-763d00ba4e56 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:00.523020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d303b65-4378-44af-8d15-763d00ba4e56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:00.524773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7d303b65-4378-44af-8d15-763d00ba4e56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:00.551080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:00.626230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d303b65-4378-44af-8d15-763d00ba4e56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:00.627970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d303b65-4378-44af-8d15-763d00ba4e56 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:00.650223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 685/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:03.139006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e1326-6195-473a-8f04-d809bfc8593c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:03.139608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e1326-6195-473a-8f04-d809bfc8593c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:03.139608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e1326-6195-473a-8f04-d809bfc8593c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:03.141373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 679/1364] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:03.146883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a5217-5a38-402f-955f-f61bf33582b1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:03.147537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a5217-5a38-402f-955f-f61bf33582b1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:03.147537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a5217-5a38-402f-955f-f61bf33582b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:03.225172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ee7a5217-5a38-402f-955f-f61bf33582b1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:03.226428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ee7a5217-5a38-402f-955f-f61bf33582b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:03.337048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:03.348290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 686/1365] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:03.643903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8e80d-82d9-45b4-9867-14c36ce5c89e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:03.644270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8e80d-82d9-45b4-9867-14c36ce5c89e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:03.644541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8e80d-82d9-45b4-9867-14c36ce5c89e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:03.645930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 680/1366] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 13:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:03.651572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec001f7-b568-47e9-baeb-722fb99558ac None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:03.651915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec001f7-b568-47e9-baeb-722fb99558ac None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:03.658867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec001f7-b568-47e9-baeb-722fb99558ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:03.738785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2ec001f7-b568-47e9-baeb-722fb99558ac None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:03.739883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2ec001f7-b568-47e9-baeb-722fb99558ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:03.890890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 687/1367] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 13:31:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:31:03.898915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8e80d-82d9-45b4-9867-14c36ce5c89e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:03.900606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-54f8e80d-82d9-45b4-9867-14c36ce5c89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:04.074019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8e80d-82d9-45b4-9867-14c36ce5c89e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:04.098650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:04.212249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5ce38-806b-40fb-aba6-89a42d350b39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:04.212930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ce38-806b-40fb-aba6-89a42d350b39 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:04.213191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ce38-806b-40fb-aba6-89a42d350b39 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:04.213409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ce38-806b-40fb-aba6-89a42d350b39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:04.215053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a0b5ce38-806b-40fb-aba6-89a42d350b39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:04.321493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5ce38-806b-40fb-aba6-89a42d350b39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:04.323112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5ce38-806b-40fb-aba6-89a42d350b39 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:04.343969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 681/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:04.914617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66bb6-0995-4ab1-a8c0-c4b4d49228e6 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:04.915142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66bb6-0995-4ab1-a8c0-c4b4d49228e6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:04.915142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66bb6-0995-4ab1-a8c0-c4b4d49228e6 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:04.916460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 688/1369] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:04.923221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6388a2-f669-48d3-aa33-794b122dd41a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:04.923658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6388a2-f669-48d3-aa33-794b122dd41a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:04.924031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6388a2-f669-48d3-aa33-794b122dd41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:04.960442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:05.001942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7e6388a2-f669-48d3-aa33-794b122dd41a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:05.003163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7e6388a2-f669-48d3-aa33-794b122dd41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:05.124583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 682/1370] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:05.723334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de303ec8-3fa1-46b4-adac-b1468ca72c52 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:05.723334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de303ec8-3fa1-46b4-adac-b1468ca72c52 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:05.723787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de303ec8-3fa1-46b4-adac-b1468ca72c52 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:05.725039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 689/1371] 10.222.0.73 () {54 vars in 964 bytes} [Wed Aug 24 13:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:05.730498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9275c0-e135-4d53-9fdc-d4db52ff25c4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:05.732151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9275c0-e135-4d53-9fdc-d4db52ff25c4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:05.732151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9275c0-e135-4d53-9fdc-d4db52ff25c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:05.820504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8d9275c0-e135-4d53-9fdc-d4db52ff25c4 None None] MFA Rules not processed for user `2550e85fa53d416887bff88fbff937d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:05.821721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8d9275c0-e135-4d53-9fdc-d4db52ff25c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:05.871200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:05.902154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 683/1372] 10.222.0.73 () {58 vars in 1061 bytes} [Wed Aug 24 13:31:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:05.910368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de303ec8-3fa1-46b4-adac-b1468ca72c52 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:05.911893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-de303ec8-3fa1-46b4-adac-b1468ca72c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:05.965187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:06.029674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de303ec8-3fa1-46b4-adac-b1468ca72c52 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:06.109293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2984d43-f873-42fd-a383-f095041f6084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:06.109966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984d43-f873-42fd-a383-f095041f6084 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:06.110160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984d43-f873-42fd-a383-f095041f6084 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:06.110437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984d43-f873-42fd-a383-f095041f6084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:06.112066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a2984d43-f873-42fd-a383-f095041f6084 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:06.218296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:06.233908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2984d43-f873-42fd-a383-f095041f6084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:06.234420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2984d43-f873-42fd-a383-f095041f6084 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:06.266464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 690/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:06.585249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11008b-d183-4370-a860-4aafb8dcd324 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:06.585565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11008b-d183-4370-a860-4aafb8dcd324 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:06.585811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11008b-d183-4370-a860-4aafb8dcd324 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:06.590206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 684/1374] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:06.594170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d06822-eda8-42a8-b8ed-33a711ab77b3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:06.594461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d06822-eda8-42a8-b8ed-33a711ab77b3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:06.594737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d06822-eda8-42a8-b8ed-33a711ab77b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:06.666170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-78d06822-eda8-42a8-b8ed-33a711ab77b3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:06.667382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-78d06822-eda8-42a8-b8ed-33a711ab77b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:06.777513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 691/1375] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:07.075856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11008b-d183-4370-a860-4aafb8dcd324 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:07.077743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fd11008b-d183-4370-a860-4aafb8dcd324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:07.234889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11008b-d183-4370-a860-4aafb8dcd324 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:07.296336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:07.362229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7b45f-f2ec-4c0f-8b4d-8208b1377082 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:07.362849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7b45f-f2ec-4c0f-8b4d-8208b1377082 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:07.363127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7b45f-f2ec-4c0f-8b4d-8208b1377082 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:07.363397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7b45f-f2ec-4c0f-8b4d-8208b1377082 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:07.365088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c2d7b45f-f2ec-4c0f-8b4d-8208b1377082 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:07.464757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7b45f-f2ec-4c0f-8b4d-8208b1377082 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:07.467002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7b45f-f2ec-4c0f-8b4d-8208b1377082 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:07.486481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 685/1376] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:31:09.635104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f697df-2782-49fd-9862-4e605dc07fb7 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:09.635618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f697df-2782-49fd-9862-4e605dc07fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:09.635618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f697df-2782-49fd-9862-4e605dc07fb7 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:09.636973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 692/1377] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:09.642611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebe639-3ab4-401b-98e8-2d10cef71487 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:09.643098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebe639-3ab4-401b-98e8-2d10cef71487 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:09.643218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebe639-3ab4-401b-98e8-2d10cef71487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:09.710684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bdebe639-3ab4-401b-98e8-2d10cef71487 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:09.712162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bdebe639-3ab4-401b-98e8-2d10cef71487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:09.720885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:09.826997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 686/1378] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:10.136070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f697df-2782-49fd-9862-4e605dc07fb7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:10.138165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-88f697df-2782-49fd-9862-4e605dc07fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:10.207894 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:10.307248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f697df-2782-49fd-9862-4e605dc07fb7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:10.442739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274aa8-db2d-4c11-b2f5-509a2e02c86f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:10.443347 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274aa8-db2d-4c11-b2f5-509a2e02c86f service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:10.443626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274aa8-db2d-4c11-b2f5-509a2e02c86f service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:10.443896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274aa8-db2d-4c11-b2f5-509a2e02c86f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:10.445621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cc274aa8-db2d-4c11-b2f5-509a2e02c86f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:10.466984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:10.549371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274aa8-db2d-4c11-b2f5-509a2e02c86f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:10.550987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274aa8-db2d-4c11-b2f5-509a2e02c86f service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:10.571774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 693/1379] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:12.749672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20e56-8d6c-40d4-babe-5e509ece0aec None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:12.750197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20e56-8d6c-40d4-babe-5e509ece0aec None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:12.750197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20e56-8d6c-40d4-babe-5e509ece0aec None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:12.751462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 687/1380] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:12.757193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518629f-708f-4350-b155-b046b02408b8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:12.757646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518629f-708f-4350-b155-b046b02408b8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:12.757736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518629f-708f-4350-b155-b046b02408b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:12.839742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6518629f-708f-4350-b155-b046b02408b8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:12.842083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6518629f-708f-4350-b155-b046b02408b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:12.956821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:12.963791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 694/1381] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:13.272982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20e56-8d6c-40d4-babe-5e509ece0aec None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:13.274564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cfa20e56-8d6c-40d4-babe-5e509ece0aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:13.449553 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:13.460955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20e56-8d6c-40d4-babe-5e509ece0aec None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:13.596758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8fda9-978b-4a78-a5aa-93ac1036dc9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:13.597448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8fda9-978b-4a78-a5aa-93ac1036dc9f service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:13.597674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8fda9-978b-4a78-a5aa-93ac1036dc9f service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:13.597946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8fda9-978b-4a78-a5aa-93ac1036dc9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:13.599565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c5a8fda9-978b-4a78-a5aa-93ac1036dc9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:13.708932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8fda9-978b-4a78-a5aa-93ac1036dc9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:13.710546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8fda9-978b-4a78-a5aa-93ac1036dc9f service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:13.727764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:13.737999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 688/1382] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:15.909179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44499a-28f0-431b-a9f8-38685d0f10a7 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:15.909752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44499a-28f0-431b-a9f8-38685d0f10a7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:15.909752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44499a-28f0-431b-a9f8-38685d0f10a7 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:15.911091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 695/1383] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:15.917160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f611d-1571-4bf1-af4e-5804d350190b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:15.917783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f611d-1571-4bf1-af4e-5804d350190b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:15.917887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f611d-1571-4bf1-af4e-5804d350190b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:16.004647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f89f611d-1571-4bf1-af4e-5804d350190b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:16.006003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f89f611d-1571-4bf1-af4e-5804d350190b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:16.177441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 689/1384] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:31:16.589383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44499a-28f0-431b-a9f8-38685d0f10a7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:16.591461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3e44499a-28f0-431b-a9f8-38685d0f10a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:16.776513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44499a-28f0-431b-a9f8-38685d0f10a7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:16.864472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:16.918766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f77ba0-51fc-436d-93a4-7d8b0227458c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:16.919639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f77ba0-51fc-436d-93a4-7d8b0227458c service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:16.919751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f77ba0-51fc-436d-93a4-7d8b0227458c service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:16.920028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f77ba0-51fc-436d-93a4-7d8b0227458c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:16.921744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-43f77ba0-51fc-436d-93a4-7d8b0227458c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:17.040536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f77ba0-51fc-436d-93a4-7d8b0227458c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:17.042612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f77ba0-51fc-436d-93a4-7d8b0227458c service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:17.069223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 696/1385] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:19.160600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2bd4-9c4d-4794-abb5-2796ba4fb1f4 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:19.161140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2bd4-9c4d-4794-abb5-2796ba4fb1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:19.161376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2bd4-9c4d-4794-abb5-2796ba4fb1f4 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:19.163097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 690/1386] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:19.170653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ab20-282f-471a-bed5-85c9f661f903 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:19.171334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ab20-282f-471a-bed5-85c9f661f903 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:19.171334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9ab20-282f-471a-bed5-85c9f661f903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:19.245101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b4e9ab20-282f-471a-bed5-85c9f661f903 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:19.246911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b4e9ab20-282f-471a-bed5-85c9f661f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:19.290698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:19.362212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 697/1387] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:19.737161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b2bd4-9c4d-4794-abb5-2796ba4fb1f4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:19.738702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0f6b2bd4-9c4d-4794-abb5-2796ba4fb1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:19.833044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:19.927738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b2bd4-9c4d-4794-abb5-2796ba4fb1f4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:20.081584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107c7c2-35a6-4f07-942c-e8eb41e511a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:20.082198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c7c2-35a6-4f07-942c-e8eb41e511a5 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:20.082481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c7c2-35a6-4f07-942c-e8eb41e511a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:20.082779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107c7c2-35a6-4f07-942c-e8eb41e511a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:20.084427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b107c7c2-35a6-4f07-942c-e8eb41e511a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:20.128313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:20.188443 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107c7c2-35a6-4f07-942c-e8eb41e511a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:20.191151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107c7c2-35a6-4f07-942c-e8eb41e511a5 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:20.212515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 691/1388] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:22.033325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63eab8-1456-4757-9e09-a1b72f39db63 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:22.034097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63eab8-1456-4757-9e09-a1b72f39db63 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:22.034097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63eab8-1456-4757-9e09-a1b72f39db63 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:22.035403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 698/1389] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:22.041708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2000-606e-4d52-8c46-21617c9d890a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:22.042221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2000-606e-4d52-8c46-21617c9d890a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:22.042339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa2000-606e-4d52-8c46-21617c9d890a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:22.118259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-50aa2000-606e-4d52-8c46-21617c9d890a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:22.120116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-50aa2000-606e-4d52-8c46-21617c9d890a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:22.231907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 692/1390] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:22.370666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac1fee-60da-410a-87c1-1d91f02942c6 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:22.370920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac1fee-60da-410a-87c1-1d91f02942c6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:22.371584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac1fee-60da-410a-87c1-1d91f02942c6 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:22.373289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 699/1391] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:22.381018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b1f32-302f-4695-a010-49ea90611687 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:22.381325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b1f32-302f-4695-a010-49ea90611687 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:22.381744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b1f32-302f-4695-a010-49ea90611687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:22.399861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:22.458884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4e7b1f32-302f-4695-a010-49ea90611687 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:22.460113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4e7b1f32-302f-4695-a010-49ea90611687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:22.536633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2adf3-9131-4e7d-90b6-fe782e61cfbb None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:22.537017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2adf3-9131-4e7d-90b6-fe782e61cfbb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:22.537369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2adf3-9131-4e7d-90b6-fe782e61cfbb None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:22.538835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 700/1392] 10.222.0.73 () {54 vars in 967 bytes} [Wed Aug 24 13:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:31:22.545446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d184d3-b717-47e3-8717-c00a76214d05 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:22.545920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d184d3-b717-47e3-8717-c00a76214d05 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:22.546191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d184d3-b717-47e3-8717-c00a76214d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:22.569709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 693/1393] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:22.631614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d7d184d3-b717-47e3-8717-c00a76214d05 None None] MFA Rules not processed for user `d5b5c6fcfde34c05a2fec0659dfd54a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:22.632801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d7d184d3-b717-47e3-8717-c00a76214d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:22.719971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 701/1394] 10.222.0.73 () {58 vars in 1064 bytes} [Wed Aug 24 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:31:22.729896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b1f32-302f-4695-a010-49ea90611687 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:22.731203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4e7b1f32-302f-4695-a010-49ea90611687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:22.827907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b1f32-302f-4695-a010-49ea90611687 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:22.836097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:22.864395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d184d3-b717-47e3-8717-c00a76214d05 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:22.865704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d7d184d3-b717-47e3-8717-c00a76214d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:22.898197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:22.910857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703bdd9-4b9c-4df7-9022-df6e53b40e41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5c6fcfde34c05a2fec0659dfd54a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:22.911468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703bdd9-4b9c-4df7-9022-df6e53b40e41 service glance] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:22.911744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703bdd9-4b9c-4df7-9022-df6e53b40e41 service glance] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:22.912015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703bdd9-4b9c-4df7-9022-df6e53b40e41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:22.913689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c703bdd9-4b9c-4df7-9022-df6e53b40e41 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:22.999630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d184d3-b717-47e3-8717-c00a76214d05 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:23.030611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703bdd9-4b9c-4df7-9022-df6e53b40e41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:23.032066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703bdd9-4b9c-4df7-9022-df6e53b40e41 service glance] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:23.052491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 694/1395] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:31:23.118385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24755f8-550a-4244-a86a-cbe674dbeafc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:23.118985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24755f8-550a-4244-a86a-cbe674dbeafc service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:23.119241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24755f8-550a-4244-a86a-cbe674dbeafc service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:23.119514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24755f8-550a-4244-a86a-cbe674dbeafc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:23.121343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a24755f8-550a-4244-a86a-cbe674dbeafc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:23.140572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:23.218037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24755f8-550a-4244-a86a-cbe674dbeafc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:23.219711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24755f8-550a-4244-a86a-cbe674dbeafc service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:23.239069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 702/1396] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:31:23.983320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f39eb-a19f-498b-943b-7b9ff9bc68f2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:23.983640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f39eb-a19f-498b-943b-7b9ff9bc68f2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:23.983918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f39eb-a19f-498b-943b-7b9ff9bc68f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:24.034521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:24.058656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-659f39eb-a19f-498b-943b-7b9ff9bc68f2 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:24.060193 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-659f39eb-a19f-498b-943b-7b9ff9bc68f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:24.151443 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 695/1397] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:24.198415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de0e0-cfda-4817-94e3-3e4007d94fed None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:24.198913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de0e0-cfda-4817-94e3-3e4007d94fed None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:24.199746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de0e0-cfda-4817-94e3-3e4007d94fed None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:24.201621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 703/1398] 10.222.0.73 () {54 vars in 943 bytes} [Wed Aug 24 13:31:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:24.208675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d3ff8-6417-4a20-96df-8b272a10b8fe None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:24.208969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d3ff8-6417-4a20-96df-8b272a10b8fe None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:24.209244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d3ff8-6417-4a20-96df-8b272a10b8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:24.288902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-081d3ff8-6417-4a20-96df-8b272a10b8fe None None] MFA Rules not processed for user `bd886328304543828dca35df3a0b7411`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:24.290124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-081d3ff8-6417-4a20-96df-8b272a10b8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:24.354506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:24.406194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 696/1399] 10.222.0.73 () {58 vars in 1040 bytes} [Wed Aug 24 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:24.415642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de0e0-cfda-4817-94e3-3e4007d94fed None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:24.417220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-269de0e0-cfda-4817-94e3-3e4007d94fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:24.566319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de0e0-cfda-4817-94e3-3e4007d94fed None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:24.576423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:24.679206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00c147-aedb-4d64-83af-a71afee21a3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:24.679813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c147-aedb-4d64-83af-a71afee21a3d service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:24.680375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c147-aedb-4d64-83af-a71afee21a3d service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:24.680650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c147-aedb-4d64-83af-a71afee21a3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:24.682316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0b00c147-aedb-4d64-83af-a71afee21a3d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:24.769120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00c147-aedb-4d64-83af-a71afee21a3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:24.770712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00c147-aedb-4d64-83af-a71afee21a3d service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:24.779613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 704/1400] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:31:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:31:25.319149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60d66-e91d-4a4c-8203-656496ee2702 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:25.319712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60d66-e91d-4a4c-8203-656496ee2702 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:25.319712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60d66-e91d-4a4c-8203-656496ee2702 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:25.321069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 697/1401] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:31:25.327362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a066c-4fae-46b8-b0de-29e90cda6eea None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:25.327891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a066c-4fae-46b8-b0de-29e90cda6eea None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:25.327891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a066c-4fae-46b8-b0de-29e90cda6eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:25.384249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:25.404843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-102a066c-4fae-46b8-b0de-29e90cda6eea None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:25.406101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-102a066c-4fae-46b8-b0de-29e90cda6eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:25.532536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 705/1402] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:25.877148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f60d66-e91d-4a4c-8203-656496ee2702 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:25.878676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-13f60d66-e91d-4a4c-8203-656496ee2702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:26.055996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f60d66-e91d-4a4c-8203-656496ee2702 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:26.105396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:26.204273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3fc34-4c75-4c03-954c-a8178f9ebcbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:26.205390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3fc34-4c75-4c03-954c-a8178f9ebcbe service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:26.206089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3fc34-4c75-4c03-954c-a8178f9ebcbe service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:26.206363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3fc34-4c75-4c03-954c-a8178f9ebcbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:26.208500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-11a3fc34-4c75-4c03-954c-a8178f9ebcbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:26.318516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3fc34-4c75-4c03-954c-a8178f9ebcbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:26.320107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3fc34-4c75-4c03-954c-a8178f9ebcbe service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:26.340371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 698/1403] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:28.515702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153fbb8-9f0c-4aab-99d3-f0497b3c9543 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:28.516279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153fbb8-9f0c-4aab-99d3-f0497b3c9543 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:28.516279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153fbb8-9f0c-4aab-99d3-f0497b3c9543 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:28.517766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 706/1404] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:28.523498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a15d4-a25f-4c47-9edf-b8a359aeea31 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:28.523997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a15d4-a25f-4c47-9edf-b8a359aeea31 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:28.523997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a15d4-a25f-4c47-9edf-b8a359aeea31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:28.588953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:28.599234 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f42a15d4-a25f-4c47-9edf-b8a359aeea31 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:28.600799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f42a15d4-a25f-4c47-9edf-b8a359aeea31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:28.728205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 699/1405] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:29.018471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153fbb8-9f0c-4aab-99d3-f0497b3c9543 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:29.020236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9153fbb8-9f0c-4aab-99d3-f0497b3c9543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:29.154558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:29.190698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153fbb8-9f0c-4aab-99d3-f0497b3c9543 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:29.337562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978a25e-56f4-411c-8135-2ebb3c07beec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:29.338172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978a25e-56f4-411c-8135-2ebb3c07beec service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:29.338454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978a25e-56f4-411c-8135-2ebb3c07beec service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:29.338729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978a25e-56f4-411c-8135-2ebb3c07beec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:29.340464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6978a25e-56f4-411c-8135-2ebb3c07beec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:29.441050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:29.452294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978a25e-56f4-411c-8135-2ebb3c07beec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:29.457935 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978a25e-56f4-411c-8135-2ebb3c07beec service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:29.476590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 707/1406] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:31.640803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f9fa-58cb-4213-95e3-97afa22cc504 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:31.643607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f9fa-58cb-4213-95e3-97afa22cc504 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:31.644213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f9fa-58cb-4213-95e3-97afa22cc504 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:31.649984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 700/1407] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:31:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:31.653158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86c13-5442-444b-b96c-7c5ca43f1778 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:31.654489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86c13-5442-444b-b96c-7c5ca43f1778 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:31.654948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86c13-5442-444b-b96c-7c5ca43f1778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:31.738752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-dee86c13-5442-444b-b96c-7c5ca43f1778 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:31.740018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dee86c13-5442-444b-b96c-7c5ca43f1778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:31.873102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 708/1408] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:32.181029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5f9fa-58cb-4213-95e3-97afa22cc504 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:32.182568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6ff5f9fa-58cb-4213-95e3-97afa22cc504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:32.350579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:32.391341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5f9fa-58cb-4213-95e3-97afa22cc504 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:32.546890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232fe25-e9c5-44a4-b381-2305b1b90bd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:32.546890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232fe25-e9c5-44a4-b381-2305b1b90bd3 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:32.546890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232fe25-e9c5-44a4-b381-2305b1b90bd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:32.546890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232fe25-e9c5-44a4-b381-2305b1b90bd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:32.547393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1232fe25-e9c5-44a4-b381-2305b1b90bd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:32.653019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:32.656982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232fe25-e9c5-44a4-b381-2305b1b90bd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:32.658544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232fe25-e9c5-44a4-b381-2305b1b90bd3 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:32.683121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 701/1409] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:41.870833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=74 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:42.685386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:50.110562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777286-3a85-47af-b4cd-98e6a348e229 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:50.111121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777286-3a85-47af-b4cd-98e6a348e229 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:50.111121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777286-3a85-47af-b4cd-98e6a348e229 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:50.112648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 709/1410] 10.222.0.73 () {54 vars in 918 bytes} [Wed Aug 24 13:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:50.118431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1861-c209-43f0-899d-08722d6af75a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:50.118907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1861-c209-43f0-899d-08722d6af75a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:50.119081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b1861-c209-43f0-899d-08722d6af75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:50.199753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ef3b1861-c209-43f0-899d-08722d6af75a None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:50.201014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ef3b1861-c209-43f0-899d-08722d6af75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:50.309586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 702/1411] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 13:31:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:50.324837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80377a97-f00f-442a-a47b-23a21dafb778 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:50.325121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80377a97-f00f-442a-a47b-23a21dafb778 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:50.325391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80377a97-f00f-442a-a47b-23a21dafb778 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:50.326720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 710/1412] 10.222.0.73 () {54 vars in 918 bytes} [Wed Aug 24 13:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:50.331539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0eb11-9537-4ae9-98bb-8292597bb3d0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:50.331831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0eb11-9537-4ae9-98bb-8292597bb3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:50.332144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0eb11-9537-4ae9-98bb-8292597bb3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:50.407711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:50.410826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bbb0eb11-9537-4ae9-98bb-8292597bb3d0 None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:50.412052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bbb0eb11-9537-4ae9-98bb-8292597bb3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:50.533144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 703/1413] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 13:31:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:52.986617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce2f5b-acc5-4ccc-9ad7-330417003d14 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:52.987183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce2f5b-acc5-4ccc-9ad7-330417003d14 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:52.987183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce2f5b-acc5-4ccc-9ad7-330417003d14 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:52.988602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:52.994198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e966ab4-ad45-46d6-b716-6d0c4dbc832f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:52.994653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e966ab4-ad45-46d6-b716-6d0c4dbc832f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:52.994913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e966ab4-ad45-46d6-b716-6d0c4dbc832f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:53.064258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7e966ab4-ad45-46d6-b716-6d0c4dbc832f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:53.065487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7e966ab4-ad45-46d6-b716-6d0c4dbc832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:53.121956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:53.163946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:53.475216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce2f5b-acc5-4ccc-9ad7-330417003d14 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:53.476790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-22ce2f5b-acc5-4ccc-9ad7-330417003d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:53.637576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce2f5b-acc5-4ccc-9ad7-330417003d14 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:53.710188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:53.751884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bb81c-a30f-48c9-a0ef-16b984346214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:53.752596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bb81c-a30f-48c9-a0ef-16b984346214 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:53.753257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bb81c-a30f-48c9-a0ef-16b984346214 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:53.753550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bb81c-a30f-48c9-a0ef-16b984346214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:53.755165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5b4bb81c-a30f-48c9-a0ef-16b984346214 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:53.854927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bb81c-a30f-48c9-a0ef-16b984346214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:53.856575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bb81c-a30f-48c9-a0ef-16b984346214 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:53.875160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 712/1416] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:54.779350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b3641-047a-4410-864c-4b96547210d3 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:54.779934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b3641-047a-4410-864c-4b96547210d3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:54.779934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b3641-047a-4410-864c-4b96547210d3 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:54.781305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:54.786486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05f8a2-0e99-4cfd-aee3-fa3dcb8b25a9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:54.786928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05f8a2-0e99-4cfd-aee3-fa3dcb8b25a9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:54.787069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05f8a2-0e99-4cfd-aee3-fa3dcb8b25a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:54.867955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3b05f8a2-0e99-4cfd-aee3-fa3dcb8b25a9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:54.869316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3b05f8a2-0e99-4cfd-aee3-fa3dcb8b25a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:54.909302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:55.013791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:31:55.164885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b3641-047a-4410-864c-4b96547210d3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:55.167411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-789b3641-047a-4410-864c-4b96547210d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:55.326675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b3641-047a-4410-864c-4b96547210d3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:55.371645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:55.445601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53914d0d-f7fe-4618-89de-ecd7ae39dad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:55.446205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53914d0d-f7fe-4618-89de-ecd7ae39dad0 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:55.446484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53914d0d-f7fe-4618-89de-ecd7ae39dad0 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:55.446754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53914d0d-f7fe-4618-89de-ecd7ae39dad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:55.448502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-53914d0d-f7fe-4618-89de-ecd7ae39dad0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:55.541373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53914d0d-f7fe-4618-89de-ecd7ae39dad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:55.543620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53914d0d-f7fe-4618-89de-ecd7ae39dad0 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:55.563629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 706/1419] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:56.465310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226fcfa-38b1-4da3-9d5e-f3e816f9d298 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:56.465888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226fcfa-38b1-4da3-9d5e-f3e816f9d298 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:56.465888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226fcfa-38b1-4da3-9d5e-f3e816f9d298 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:56.467204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:56.472537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d17b-bb75-4c4c-9289-44f0482c7376 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:56.472983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d17b-bb75-4c4c-9289-44f0482c7376 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:56.473105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407d17b-bb75-4c4c-9289-44f0482c7376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:56.529289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:56.552361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1407d17b-bb75-4c4c-9289-44f0482c7376 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:56.553592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1407d17b-bb75-4c4c-9289-44f0482c7376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:56.654464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:56.806840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226fcfa-38b1-4da3-9d5e-f3e816f9d298 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:56.808567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0226fcfa-38b1-4da3-9d5e-f3e816f9d298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:56.960379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226fcfa-38b1-4da3-9d5e-f3e816f9d298 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:56.973071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:57.074173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65647-ada5-411c-b51a-4ced003f8a2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:57.074773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65647-ada5-411c-b51a-4ced003f8a2c service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:57.075053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65647-ada5-411c-b51a-4ced003f8a2c service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:57.075325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65647-ada5-411c-b51a-4ced003f8a2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:57.076961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-32a65647-ada5-411c-b51a-4ced003f8a2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:57.169361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a65647-ada5-411c-b51a-4ced003f8a2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:57.170988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a65647-ada5-411c-b51a-4ced003f8a2c service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:57.189622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 715/1422] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:58.101924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58113608-2565-490b-aeaa-9ebfe3fe2539 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:58.102494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58113608-2565-490b-aeaa-9ebfe3fe2539 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:58.102494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58113608-2565-490b-aeaa-9ebfe3fe2539 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:58.103828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:58.109550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406db7d-a055-4326-af3d-824b37b2a685 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:58.110023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406db7d-a055-4326-af3d-824b37b2a685 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:58.110120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406db7d-a055-4326-af3d-824b37b2a685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:58.131182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:58.318374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b406db7d-a055-4326-af3d-824b37b2a685 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:58.319611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b406db7d-a055-4326-af3d-824b37b2a685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:58.422026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:31:58.576097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58113608-2565-490b-aeaa-9ebfe3fe2539 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:31:58.577749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-58113608-2565-490b-aeaa-9ebfe3fe2539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:58.704913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:58.742613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58113608-2565-490b-aeaa-9ebfe3fe2539 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:31:58.861611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedb72e-6ef8-4bde-9588-a929bc35cd18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:31:58.862227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedb72e-6ef8-4bde-9588-a929bc35cd18 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:58.862506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedb72e-6ef8-4bde-9588-a929bc35cd18 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:58.862780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedb72e-6ef8-4bde-9588-a929bc35cd18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:58.864463 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2bedb72e-6ef8-4bde-9588-a929bc35cd18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:31:58.951192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:31:58.965489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedb72e-6ef8-4bde-9588-a929bc35cd18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:31:58.978537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedb72e-6ef8-4bde-9588-a929bc35cd18 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:31:58.986321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 709/1425] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:31:59.889465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fbfab-a5d1-4139-ae1d-24b3ff579714 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:59.890013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fbfab-a5d1-4139-ae1d-24b3ff579714 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:59.890013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fbfab-a5d1-4139-ae1d-24b3ff579714 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:59.891419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:31:59.896643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af629-09e3-4dc6-bd9d-ee9f8d87979c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:31:59.897104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af629-09e3-4dc6-bd9d-ee9f8d87979c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:31:59.897274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af629-09e3-4dc6-bd9d-ee9f8d87979c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:31:59.983271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b18af629-09e3-4dc6-bd9d-ee9f8d87979c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:31:59.984534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b18af629-09e3-4dc6-bd9d-ee9f8d87979c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:00.083194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:00.256661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fbfab-a5d1-4139-ae1d-24b3ff579714 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:00.258316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4b9fbfab-a5d1-4139-ae1d-24b3ff579714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:00.346045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:00.408401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fbfab-a5d1-4139-ae1d-24b3ff579714 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:00.526093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070fdbdc-e8d3-4bae-9401-8c9b4e57395c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:00.526697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fdbdc-e8d3-4bae-9401-8c9b4e57395c service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:00.526973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fdbdc-e8d3-4bae-9401-8c9b4e57395c service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:00.527239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070fdbdc-e8d3-4bae-9401-8c9b4e57395c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:00.528945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-070fdbdc-e8d3-4bae-9401-8c9b4e57395c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:00.583002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:00.625846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fdbdc-e8d3-4bae-9401-8c9b4e57395c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:00.627330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070fdbdc-e8d3-4bae-9401-8c9b4e57395c service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:00.645657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 718/1428] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:01.529220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915220-bed0-4af4-bec7-8b1c48220fca None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:01.530015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915220-bed0-4af4-bec7-8b1c48220fca None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:01.530015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6915220-bed0-4af4-bec7-8b1c48220fca None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:01.531114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:01.537466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfda29-b04a-48d3-87eb-5e239fbb2266 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:01.537888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfda29-b04a-48d3-87eb-5e239fbb2266 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:01.538124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfda29-b04a-48d3-87eb-5e239fbb2266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:01.629292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7cbfda29-b04a-48d3-87eb-5e239fbb2266 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:01.630781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7cbfda29-b04a-48d3-87eb-5e239fbb2266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:01.735874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:01.897917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915220-bed0-4af4-bec7-8b1c48220fca None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:01.899619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b6915220-bed0-4af4-bec7-8b1c48220fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:01.951981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:02.058302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6915220-bed0-4af4-bec7-8b1c48220fca None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:02.179719 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20cc38-b655-4e06-941d-8eac7b01ae6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:02.180274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20cc38-b655-4e06-941d-8eac7b01ae6b service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:02.180819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20cc38-b655-4e06-941d-8eac7b01ae6b service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:02.181090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20cc38-b655-4e06-941d-8eac7b01ae6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:02.183579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7c20cc38-b655-4e06-941d-8eac7b01ae6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:02.190978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:02.279214 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20cc38-b655-4e06-941d-8eac7b01ae6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:02.280875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20cc38-b655-4e06-941d-8eac7b01ae6b service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:02.300135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 712/1431] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:03.211084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62afbd4-a6b2-4cd4-93a0-ce04144a205f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:03.211654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62afbd4-a6b2-4cd4-93a0-ce04144a205f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:03.211654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62afbd4-a6b2-4cd4-93a0-ce04144a205f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:03.213008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:03.218921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eef0e-1c81-4a32-a504-583049f14f0d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:03.219405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eef0e-1c81-4a32-a504-583049f14f0d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:03.219504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eef0e-1c81-4a32-a504-583049f14f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:03.304517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d72eef0e-1c81-4a32-a504-583049f14f0d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:03.306015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d72eef0e-1c81-4a32-a504-583049f14f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:03.377843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:03.408617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:03.561481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62afbd4-a6b2-4cd4-93a0-ce04144a205f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:03.563064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b62afbd4-a6b2-4cd4-93a0-ce04144a205f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:03.582793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:03.733190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62afbd4-a6b2-4cd4-93a0-ce04144a205f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:03.836386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:03.866778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396c561-5dea-4dd2-86f2-403238dfaaee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:03.867384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c561-5dea-4dd2-86f2-403238dfaaee service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:03.867668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c561-5dea-4dd2-86f2-403238dfaaee service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:03.867999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396c561-5dea-4dd2-86f2-403238dfaaee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:03.870772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3396c561-5dea-4dd2-86f2-403238dfaaee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:03.975447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396c561-5dea-4dd2-86f2-403238dfaaee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:03.977073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396c561-5dea-4dd2-86f2-403238dfaaee service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:04.001411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 721/1434] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:04.917138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72334c-a704-4f6d-91b0-9174989708f6 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:04.917698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72334c-a704-4f6d-91b0-9174989708f6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:04.917698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72334c-a704-4f6d-91b0-9174989708f6 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:04.919056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:04.924018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a771e2-5c17-47b5-9f38-232637258643 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:04.924503 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a771e2-5c17-47b5-9f38-232637258643 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:04.924617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a771e2-5c17-47b5-9f38-232637258643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:05.003688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-56a771e2-5c17-47b5-9f38-232637258643 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:05.004835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-56a771e2-5c17-47b5-9f38-232637258643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:05.046493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:05.102141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:05.254972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72334c-a704-4f6d-91b0-9174989708f6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:05.256444 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2c72334c-a704-4f6d-91b0-9174989708f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:05.409006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72334c-a704-4f6d-91b0-9174989708f6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:05.467221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:05.537563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c79cae-4054-4ae3-8c32-bb3465c74b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:05.538177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c79cae-4054-4ae3-8c32-bb3465c74b38 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:05.538457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c79cae-4054-4ae3-8c32-bb3465c74b38 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:05.538729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c79cae-4054-4ae3-8c32-bb3465c74b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:05.540451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-19c79cae-4054-4ae3-8c32-bb3465c74b38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:05.645224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c79cae-4054-4ae3-8c32-bb3465c74b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:05.646818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c79cae-4054-4ae3-8c32-bb3465c74b38 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:05.666673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 715/1437] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:32:06.543689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ca4ad-3287-4b8e-8b67-7753ce41eb55 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:06.544294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ca4ad-3287-4b8e-8b67-7753ce41eb55 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:06.544294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ca4ad-3287-4b8e-8b67-7753ce41eb55 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:06.545659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:06.550902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698a302-5f11-49cb-9881-9963e5071ba9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:06.551389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698a302-5f11-49cb-9881-9963e5071ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:06.551503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698a302-5f11-49cb-9881-9963e5071ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:06.639148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f698a302-5f11-49cb-9881-9963e5071ba9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:06.640405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f698a302-5f11-49cb-9881-9963e5071ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:06.642503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:06.743842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:06.895117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ca4ad-3287-4b8e-8b67-7753ce41eb55 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:06.896709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-303ca4ad-3287-4b8e-8b67-7753ce41eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:07.048026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ca4ad-3287-4b8e-8b67-7753ce41eb55 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:07.076355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:07.174617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcec412-14b9-440c-9875-7a564de82742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:07.175141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcec412-14b9-440c-9875-7a564de82742 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:07.175421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcec412-14b9-440c-9875-7a564de82742 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:07.175696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcec412-14b9-440c-9875-7a564de82742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:07.177418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ffcec412-14b9-440c-9875-7a564de82742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:07.273636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcec412-14b9-440c-9875-7a564de82742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:07.275250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcec412-14b9-440c-9875-7a564de82742 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:07.296447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 724/1440] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:09.253746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f6b4-bf8c-4437-999e-ab43f9a475cc None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:09.254361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f6b4-bf8c-4437-999e-ab43f9a475cc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:09.254361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7f6b4-bf8c-4437-999e-ab43f9a475cc None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:09.255720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:09.262440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc74e1-5c31-4e59-b646-12399e2c232b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:09.262934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc74e1-5c31-4e59-b646-12399e2c232b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:09.263038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc74e1-5c31-4e59-b646-12399e2c232b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:09.309127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:09.336440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8bbc74e1-5c31-4e59-b646-12399e2c232b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:09.337715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8bbc74e1-5c31-4e59-b646-12399e2c232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:09.440134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:32:09.746226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7f6b4-bf8c-4437-999e-ab43f9a475cc None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:09.747805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7bf7f6b4-bf8c-4437-999e-ab43f9a475cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:09.860036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7f6b4-bf8c-4437-999e-ab43f9a475cc None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:09.894751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:09.943084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a99c7-0445-463b-9f23-b0696809ce3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:09.943692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a99c7-0445-463b-9f23-b0696809ce3b service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:09.943969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a99c7-0445-463b-9f23-b0696809ce3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:09.944266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a99c7-0445-463b-9f23-b0696809ce3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:09.945894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-819a99c7-0445-463b-9f23-b0696809ce3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:10.064102 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a99c7-0445-463b-9f23-b0696809ce3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:10.065599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a99c7-0445-463b-9f23-b0696809ce3b service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:10.085212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 718/1443] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:32:12.349369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e919a10-b169-4bfd-9150-b1082d11e717 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:12.350188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e919a10-b169-4bfd-9150-b1082d11e717 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:12.350188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e919a10-b169-4bfd-9150-b1082d11e717 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:12.352400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:32:12.359256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5839ca2-88a4-4113-9d6d-773180684ea3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:12.359804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5839ca2-88a4-4113-9d6d-773180684ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:12.359804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5839ca2-88a4-4113-9d6d-773180684ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:12.433820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e5839ca2-88a4-4113-9d6d-773180684ea3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:12.435055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e5839ca2-88a4-4113-9d6d-773180684ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:12.448685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:12.535516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:12.825079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e919a10-b169-4bfd-9150-b1082d11e717 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:12.826617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3e919a10-b169-4bfd-9150-b1082d11e717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:12.935794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e919a10-b169-4bfd-9150-b1082d11e717 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:12.948846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:13.014081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2656a9-6364-41c2-b39b-2ba31871b56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:13.014604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2656a9-6364-41c2-b39b-2ba31871b56e service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:13.014883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2656a9-6364-41c2-b39b-2ba31871b56e service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:13.015152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2656a9-6364-41c2-b39b-2ba31871b56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:13.016923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6a2656a9-6364-41c2-b39b-2ba31871b56e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:13.132275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2656a9-6364-41c2-b39b-2ba31871b56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:13.133647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2656a9-6364-41c2-b39b-2ba31871b56e service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:13.152918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 727/1446] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:15.943270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a32849-cdd8-4ddf-b061-22164b759f58 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:15.943956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a32849-cdd8-4ddf-b061-22164b759f58 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:15.943956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a32849-cdd8-4ddf-b061-22164b759f58 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:15.945194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:15.952036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cabeb-aaf4-4ec5-8abb-f1d2719b0f0c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:15.952596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cabeb-aaf4-4ec5-8abb-f1d2719b0f0c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:15.952596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cabeb-aaf4-4ec5-8abb-f1d2719b0f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:16.001351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:16.019687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4f5cabeb-aaf4-4ec5-8abb-f1d2719b0f0c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:16.020943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4f5cabeb-aaf4-4ec5-8abb-f1d2719b0f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:16.132303 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:16.419862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a32849-cdd8-4ddf-b061-22164b759f58 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:16.421447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f7a32849-cdd8-4ddf-b061-22164b759f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:16.534481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a32849-cdd8-4ddf-b061-22164b759f58 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:16.576295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:16.613169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7a615-b5d5-4c71-bc1d-68b59bef23b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:16.613775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7a615-b5d5-4c71-bc1d-68b59bef23b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:16.614054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7a615-b5d5-4c71-bc1d-68b59bef23b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:16.614323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7a615-b5d5-4c71-bc1d-68b59bef23b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:16.615923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-26b7a615-b5d5-4c71-bc1d-68b59bef23b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:16.732683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7a615-b5d5-4c71-bc1d-68b59bef23b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:16.734081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7a615-b5d5-4c71-bc1d-68b59bef23b7 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:16.757747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 721/1449] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:18.845733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b67b9-1c2d-4773-ad18-86a510fb835c None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:18.846278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b67b9-1c2d-4773-ad18-86a510fb835c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:18.846389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b67b9-1c2d-4773-ad18-86a510fb835c None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:18.847777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:18.854469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6608c-d808-4512-a167-fa2ea007cad6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:18.855019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6608c-d808-4512-a167-fa2ea007cad6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:18.855019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6608c-d808-4512-a167-fa2ea007cad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:18.931044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-51c6608c-d808-4512-a167-fa2ea007cad6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:18.933782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-51c6608c-d808-4512-a167-fa2ea007cad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:18.956518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:19.041469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:19.340658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b67b9-1c2d-4773-ad18-86a510fb835c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:19.342232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-466b67b9-1c2d-4773-ad18-86a510fb835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:19.463371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b67b9-1c2d-4773-ad18-86a510fb835c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:19.486274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:19.546668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fad7a-552f-43bc-90c3-96b692b191f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:19.547273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fad7a-552f-43bc-90c3-96b692b191f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:19.547549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fad7a-552f-43bc-90c3-96b692b191f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:19.547822 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fad7a-552f-43bc-90c3-96b692b191f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:19.549504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a15fad7a-552f-43bc-90c3-96b692b191f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:19.670811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fad7a-552f-43bc-90c3-96b692b191f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:19.672563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fad7a-552f-43bc-90c3-96b692b191f3 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:19.693696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 730/1452] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:21.745966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4968-9f28-449b-bd99-506a6e2e3214 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:21.746487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4968-9f28-449b-bd99-506a6e2e3214 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:21.746590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf4968-9f28-449b-bd99-506a6e2e3214 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:21.747890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:21.753849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485c7a5-b49b-4221-a52d-8013ff9e31e3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:21.754270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485c7a5-b49b-4221-a52d-8013ff9e31e3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:21.754425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485c7a5-b49b-4221-a52d-8013ff9e31e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:21.842552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:21.845896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2485c7a5-b49b-4221-a52d-8013ff9e31e3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:21.847136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2485c7a5-b49b-4221-a52d-8013ff9e31e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:21.963152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:22.251761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf4968-9f28-449b-bd99-506a6e2e3214 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:22.253418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-18cf4968-9f28-449b-bd99-506a6e2e3214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:22.380250 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf4968-9f28-449b-bd99-506a6e2e3214 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:22.440635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:22.467599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd73e60-483a-4207-a409-d58f142171d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:22.468291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73e60-483a-4207-a409-d58f142171d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:22.468625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73e60-483a-4207-a409-d58f142171d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:22.468795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd73e60-483a-4207-a409-d58f142171d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:22.470447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dbd73e60-483a-4207-a409-d58f142171d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:22.594688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd73e60-483a-4207-a409-d58f142171d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:22.596131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd73e60-483a-4207-a409-d58f142171d7 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:22.615616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 724/1455] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:32:24.702443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f4c6-4b90-4122-915f-1c8c5d2a70ef None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:24.703079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f4c6-4b90-4122-915f-1c8c5d2a70ef None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:24.703079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f4c6-4b90-4122-915f-1c8c5d2a70ef None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:24.704488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:24.710356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af002-85ba-40bd-8435-faf5eaf86ad3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:24.710869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af002-85ba-40bd-8435-faf5eaf86ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:24.711012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af002-85ba-40bd-8435-faf5eaf86ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:24.788990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-558af002-85ba-40bd-8435-faf5eaf86ad3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:24.790249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-558af002-85ba-40bd-8435-faf5eaf86ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:24.823490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:24.903067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:25.194999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f4c6-4b90-4122-915f-1c8c5d2a70ef None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:25.196777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ca62f4c6-4b90-4122-915f-1c8c5d2a70ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:25.313565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f4c6-4b90-4122-915f-1c8c5d2a70ef None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:25.342903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:25.391307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f633db-c534-4f7b-ae18-4ed680abfebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:25.391922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f633db-c534-4f7b-ae18-4ed680abfebd service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:25.392220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f633db-c534-4f7b-ae18-4ed680abfebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:25.392506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f633db-c534-4f7b-ae18-4ed680abfebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:25.394111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d1f633db-c534-4f7b-ae18-4ed680abfebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:25.506258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f633db-c534-4f7b-ae18-4ed680abfebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:25.507991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f633db-c534-4f7b-ae18-4ed680abfebd service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:25.536557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 733/1458] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:27.499190 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2579e-397f-42dc-b65d-3191caaad55e None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:27.499869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2579e-397f-42dc-b65d-3191caaad55e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:27.499869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2579e-397f-42dc-b65d-3191caaad55e None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:27.501114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:27.507101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11a25-3193-4d88-a0f5-73ef3a8457fb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:27.507584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11a25-3193-4d88-a0f5-73ef3a8457fb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:27.507719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c11a25-3193-4d88-a0f5-73ef3a8457fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:27.576680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-48c11a25-3193-4d88-a0f5-73ef3a8457fb None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:27.577904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-48c11a25-3193-4d88-a0f5-73ef3a8457fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:27.588086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:27.688333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:27.971792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2579e-397f-42dc-b65d-3191caaad55e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:27.973367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-98c2579e-397f-42dc-b65d-3191caaad55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:28.080689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2579e-397f-42dc-b65d-3191caaad55e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:28.137303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:28.156341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdbcc8-04cd-4fa7-801e-866d7cae1780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:28.156954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdbcc8-04cd-4fa7-801e-866d7cae1780 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:28.157238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdbcc8-04cd-4fa7-801e-866d7cae1780 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:28.157512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdbcc8-04cd-4fa7-801e-866d7cae1780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:28.159125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-40fdbcc8-04cd-4fa7-801e-866d7cae1780 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:28.275017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdbcc8-04cd-4fa7-801e-866d7cae1780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:28.276694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdbcc8-04cd-4fa7-801e-866d7cae1780 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:28.295320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 727/1461] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:30.312792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e0564-5052-4315-a492-fec04e6f90c8 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:30.313361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e0564-5052-4315-a492-fec04e6f90c8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:30.313361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e0564-5052-4315-a492-fec04e6f90c8 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:30.314703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:30.320150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14a3c-ac85-4879-8342-c546967e0349 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:30.320599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14a3c-ac85-4879-8342-c546967e0349 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:30.320746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c14a3c-ac85-4879-8342-c546967e0349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:30.393914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c6c14a3c-ac85-4879-8342-c546967e0349 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:30.395143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c6c14a3c-ac85-4879-8342-c546967e0349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:30.435524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:30.494618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:30.774952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e0564-5052-4315-a492-fec04e6f90c8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:30.776517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e37e0564-5052-4315-a492-fec04e6f90c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:30.882801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e0564-5052-4315-a492-fec04e6f90c8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:30.923211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:30.962033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f6910-ac96-4ace-941e-4665725c2fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:30.962841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f6910-ac96-4ace-941e-4665725c2fac service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:30.963121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f6910-ac96-4ace-941e-4665725c2fac service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:30.963395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f6910-ac96-4ace-941e-4665725c2fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:30.965027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-398f6910-ac96-4ace-941e-4665725c2fac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:31.073161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f6910-ac96-4ace-941e-4665725c2fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:31.074588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f6910-ac96-4ace-941e-4665725c2fac service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:31.092577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 736/1464] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:33.089247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21e9b5-3078-43fd-a149-09f57c8be447 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:33.089806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21e9b5-3078-43fd-a149-09f57c8be447 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:33.089806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21e9b5-3078-43fd-a149-09f57c8be447 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:33.091160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:33.096645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a57dd-98ad-4592-b78c-775edf1fb3a0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:33.097103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a57dd-98ad-4592-b78c-775edf1fb3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:33.097216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a57dd-98ad-4592-b78c-775edf1fb3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:33.163352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-044a57dd-98ad-4592-b78c-775edf1fb3a0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:33.164608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-044a57dd-98ad-4592-b78c-775edf1fb3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:33.183032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:33.264250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:33.549775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21e9b5-3078-43fd-a149-09f57c8be447 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:33.551398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4f21e9b5-3078-43fd-a149-09f57c8be447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:33.660819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21e9b5-3078-43fd-a149-09f57c8be447 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:33.728045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:33.735297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50fde6-d9be-4d56-9c55-7572dfa46ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:33.735901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50fde6-d9be-4d56-9c55-7572dfa46ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:33.736191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50fde6-d9be-4d56-9c55-7572dfa46ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:33.736473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50fde6-d9be-4d56-9c55-7572dfa46ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:33.738078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7d50fde6-d9be-4d56-9c55-7572dfa46ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:33.855072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50fde6-d9be-4d56-9c55-7572dfa46ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:33.856464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50fde6-d9be-4d56-9c55-7572dfa46ae0 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:33.875506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 730/1467] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:35.837981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31539f4-31f0-4e5a-8525-781cd107847b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:35.838507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31539f4-31f0-4e5a-8525-781cd107847b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:35.838631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31539f4-31f0-4e5a-8525-781cd107847b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:35.839898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:35.845433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac44f0-2a51-48f1-be62-6f884d8bed73 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:35.845883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac44f0-2a51-48f1-be62-6f884d8bed73 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:35.845991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac44f0-2a51-48f1-be62-6f884d8bed73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:35.911689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d8ac44f0-2a51-48f1-be62-6f884d8bed73 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:35.912832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d8ac44f0-2a51-48f1-be62-6f884d8bed73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:35.962914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:36.031093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:36.316482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31539f4-31f0-4e5a-8525-781cd107847b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:36.318003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e31539f4-31f0-4e5a-8525-781cd107847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:36.421415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31539f4-31f0-4e5a-8525-781cd107847b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:36.471718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:36.499603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa1caa-43fc-4cd1-b15c-e6bf824d802f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:36.500239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa1caa-43fc-4cd1-b15c-e6bf824d802f service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:36.500524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa1caa-43fc-4cd1-b15c-e6bf824d802f service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:36.500797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa1caa-43fc-4cd1-b15c-e6bf824d802f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:36.502404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c2fa1caa-43fc-4cd1-b15c-e6bf824d802f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:36.623611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa1caa-43fc-4cd1-b15c-e6bf824d802f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:36.624997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa1caa-43fc-4cd1-b15c-e6bf824d802f service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:36.643867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 739/1470] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:32:46.029243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=22 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:46.641434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=66 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:56.868954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154301e-3b87-43ec-9e87-dcc8344c874c None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:56.869747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154301e-3b87-43ec-9e87-dcc8344c874c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:56.869747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154301e-3b87-43ec-9e87-dcc8344c874c None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:56.870846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 732/1471] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:32:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:32:56.876985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6f71-7449-4936-9b37-e620873d3e6d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:56.883654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6f71-7449-4936-9b37-e620873d3e6d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:56.883654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc6f71-7449-4936-9b37-e620873d3e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:56.964118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fccc6f71-7449-4936-9b37-e620873d3e6d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:56.965683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fccc6f71-7449-4936-9b37-e620873d3e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:57.076970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 740/1472] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:57.397126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351d881-dd55-4777-969f-09498c4e4e02 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:57.397408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351d881-dd55-4777-969f-09498c4e4e02 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:57.397684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351d881-dd55-4777-969f-09498c4e4e02 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:57.399404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 733/1473] 10.222.0.73 () {54 vars in 967 bytes} [Wed Aug 24 13:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:32:57.404973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8e56-02a7-4d55-a8ba-ce006a7809b7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:57.409409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8e56-02a7-4d55-a8ba-ce006a7809b7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:57.409409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8e56-02a7-4d55-a8ba-ce006a7809b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:57.507942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-63ff8e56-02a7-4d55-a8ba-ce006a7809b7 None None] MFA Rules not processed for user `d5b5c6fcfde34c05a2fec0659dfd54a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:32:57.510148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-63ff8e56-02a7-4d55-a8ba-ce006a7809b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:57.532895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:57.591024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 741/1474] 10.222.0.73 () {58 vars in 1064 bytes} [Wed Aug 24 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:32:57.598898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351d881-dd55-4777-969f-09498c4e4e02 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:32:57.600480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4351d881-dd55-4777-969f-09498c4e4e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:57.712161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351d881-dd55-4777-969f-09498c4e4e02 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:32:57.789764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30285fb4-0630-4fae-ab67-6deeddc516a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5c6fcfde34c05a2fec0659dfd54a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:32:57.790603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30285fb4-0630-4fae-ab67-6deeddc516a9 service glance] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:32:57.790888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30285fb4-0630-4fae-ab67-6deeddc516a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:32:57.791156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30285fb4-0630-4fae-ab67-6deeddc516a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:32:57.793214 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-30285fb4-0630-4fae-ab67-6deeddc516a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:32:57.858131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:32:57.913437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30285fb4-0630-4fae-ab67-6deeddc516a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:32:57.915012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30285fb4-0630-4fae-ab67-6deeddc516a9 service glance] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:32:57.933680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 734/1475] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:33:00.105286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991982e3-44db-4d21-a6e9-d0ba80d64c5a None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:00.105867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991982e3-44db-4d21-a6e9-d0ba80d64c5a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:00.105867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991982e3-44db-4d21-a6e9-d0ba80d64c5a None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:00.107192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 742/1476] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:33:00.115396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155200-6c41-41ec-a719-dad62ac0e14a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:00.115923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155200-6c41-41ec-a719-dad62ac0e14a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:00.116035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c155200-6c41-41ec-a719-dad62ac0e14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:00.222333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1c155200-6c41-41ec-a719-dad62ac0e14a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:33:00.223644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1c155200-6c41-41ec-a719-dad62ac0e14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:00.352141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 735/1477] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:33:00.707017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991982e3-44db-4d21-a6e9-d0ba80d64c5a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:33:00.708585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-991982e3-44db-4d21-a6e9-d0ba80d64c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:00.871746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991982e3-44db-4d21-a6e9-d0ba80d64c5a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:33:00.948040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:01.008633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204e467-14e1-4a7c-aac5-050fa0fde80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:33:01.009150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e467-14e1-4a7c-aac5-050fa0fde80a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:01.009426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e467-14e1-4a7c-aac5-050fa0fde80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:01.009688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204e467-14e1-4a7c-aac5-050fa0fde80a admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:01.010554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204e467-14e1-4a7c-aac5-050fa0fde80a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:33:01.012492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-8204e467-14e1-4a7c-aac5-050fa0fde80a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:33:01.012492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Aug 24 13:33:01.012492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: Aug 24 13:33:01.013631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8204e467-14e1-4a7c-aac5-050fa0fde80a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:33:01.035326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 743/1478] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 13:33:00 2022] GET /identity/v3/projects => generated 2553 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:33:03.216012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da83b9-30d8-435a-86ba-09456e504e31 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:03.216636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da83b9-30d8-435a-86ba-09456e504e31 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:03.216636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da83b9-30d8-435a-86ba-09456e504e31 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:03.217965 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 736/1479] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:33:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:33:03.223483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158f0c-e2ca-4e75-af55-dbce1b504d50 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:03.223955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158f0c-e2ca-4e75-af55-dbce1b504d50 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:03.224091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02158f0c-e2ca-4e75-af55-dbce1b504d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:03.303764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-02158f0c-e2ca-4e75-af55-dbce1b504d50 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:33:03.305220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-02158f0c-e2ca-4e75-af55-dbce1b504d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:03.403250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 744/1480] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:33:03.700473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da83b9-30d8-435a-86ba-09456e504e31 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:33:03.702013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-20da83b9-30d8-435a-86ba-09456e504e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:03.712414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:03.868929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da83b9-30d8-435a-86ba-09456e504e31 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:33:03.952948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:03.987251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b72c1-510d-4466-8e33-10c75c98e5bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:33:03.987855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b72c1-510d-4466-8e33-10c75c98e5bd service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:03.988141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b72c1-510d-4466-8e33-10c75c98e5bd service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:03.988445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b72c1-510d-4466-8e33-10c75c98e5bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:03.990070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4b4b72c1-510d-4466-8e33-10c75c98e5bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:04.081257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b72c1-510d-4466-8e33-10c75c98e5bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:33:04.082870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b72c1-510d-4466-8e33-10c75c98e5bd service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:33:04.100716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 737/1481] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:33:06.242233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b964f-3741-49d2-af4d-fe225dd6f7cd None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:06.242821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b964f-3741-49d2-af4d-fe225dd6f7cd None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:06.242821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b964f-3741-49d2-af4d-fe225dd6f7cd None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:06.244128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 745/1482] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:33:06.249622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cae68a-d68f-43a5-9a31-e2d6e40a23c5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:06.250066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cae68a-d68f-43a5-9a31-e2d6e40a23c5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:06.250164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cae68a-d68f-43a5-9a31-e2d6e40a23c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:06.316262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-81cae68a-d68f-43a5-9a31-e2d6e40a23c5 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:33:06.317476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-81cae68a-d68f-43a5-9a31-e2d6e40a23c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:06.371491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:06.423150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 738/1483] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:33:06.710274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b964f-3741-49d2-af4d-fe225dd6f7cd None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:33:06.712301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-481b964f-3741-49d2-af4d-fe225dd6f7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:06.735956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:06.818278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b964f-3741-49d2-af4d-fe225dd6f7cd None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:33:06.920249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c64b9-f922-4742-b87c-0498d407bf81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:33:06.920863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c64b9-f922-4742-b87c-0498d407bf81 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:06.921153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c64b9-f922-4742-b87c-0498d407bf81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:06.921421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c64b9-f922-4742-b87c-0498d407bf81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:06.923082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-257c64b9-f922-4742-b87c-0498d407bf81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:07.009102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:07.047206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c64b9-f922-4742-b87c-0498d407bf81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:33:07.048688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c64b9-f922-4742-b87c-0498d407bf81 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:33:07.066955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 746/1484] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:33:09.144068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27c9c-6c1b-4ca5-bc86-386e5a2be6ff None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:09.144656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27c9c-6c1b-4ca5-bc86-386e5a2be6ff None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:09.144656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27c9c-6c1b-4ca5-bc86-386e5a2be6ff None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:09.145993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 739/1485] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:33:09.152078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de85dd9-9fbb-4d9e-bf83-b5b594ed7531 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:09.152586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de85dd9-9fbb-4d9e-bf83-b5b594ed7531 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:09.152699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de85dd9-9fbb-4d9e-bf83-b5b594ed7531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:09.220176 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6de85dd9-9fbb-4d9e-bf83-b5b594ed7531 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:33:09.221461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6de85dd9-9fbb-4d9e-bf83-b5b594ed7531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:09.319010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 747/1486] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:33:09.609877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27c9c-6c1b-4ca5-bc86-386e5a2be6ff None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:33:09.611452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dbe27c9c-6c1b-4ca5-bc86-386e5a2be6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:09.715609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27c9c-6c1b-4ca5-bc86-386e5a2be6ff None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:33:09.787876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:09.790386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4d8b0-ec13-4ea7-8987-83270a41e671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:33:09.790990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4d8b0-ec13-4ea7-8987-83270a41e671 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:09.791272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4d8b0-ec13-4ea7-8987-83270a41e671 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:09.791548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4d8b0-ec13-4ea7-8987-83270a41e671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:09.793218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d0b4d8b0-ec13-4ea7-8987-83270a41e671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:09.906538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4d8b0-ec13-4ea7-8987-83270a41e671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:33:09.908042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4d8b0-ec13-4ea7-8987-83270a41e671 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:33:09.927192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 740/1487] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:33:11.971209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e83993-3938-4998-b9ef-f058eb56fb39 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:11.971791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e83993-3938-4998-b9ef-f058eb56fb39 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:11.971791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e83993-3938-4998-b9ef-f058eb56fb39 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:11.973244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 748/1488] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:33:11.979325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013abb69-1c83-4db1-9662-899fdafc5051 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:11.979808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013abb69-1c83-4db1-9662-899fdafc5051 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:11.979915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013abb69-1c83-4db1-9662-899fdafc5051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:12.046235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-013abb69-1c83-4db1-9662-899fdafc5051 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:33:12.047811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-013abb69-1c83-4db1-9662-899fdafc5051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:12.099779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:12.144538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 741/1489] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:33:12.442107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e83993-3938-4998-b9ef-f058eb56fb39 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:33:12.443806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c5e83993-3938-4998-b9ef-f058eb56fb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:12.482524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:12.555225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e83993-3938-4998-b9ef-f058eb56fb39 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:33:12.632251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3f454-50de-4526-b619-42466428c9f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:33:12.633060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3f454-50de-4526-b619-42466428c9f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:12.633540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3f454-50de-4526-b619-42466428c9f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:12.634026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3f454-50de-4526-b619-42466428c9f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:12.635897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8ae3f454-50de-4526-b619-42466428c9f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:12.717438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:12.747840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3f454-50de-4526-b619-42466428c9f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:33:12.749840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3f454-50de-4526-b619-42466428c9f4 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:33:12.768959 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 749/1490] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:33:15.102399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53942ee9-34c0-4ade-b0c8-9a3a73d4099f None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:15.102946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53942ee9-34c0-4ade-b0c8-9a3a73d4099f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:15.102946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53942ee9-34c0-4ade-b0c8-9a3a73d4099f None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:15.104306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 742/1491] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 13:33:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:33:15.109795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83232-dc5f-4753-a6d8-e907042b9014 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:15.110240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83232-dc5f-4753-a6d8-e907042b9014 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:15.110358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa83232-dc5f-4753-a6d8-e907042b9014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:15.186763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-efa83232-dc5f-4753-a6d8-e907042b9014 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:33:15.188006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-efa83232-dc5f-4753-a6d8-e907042b9014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:15.292398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 750/1492] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 13:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:33:15.581688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53942ee9-34c0-4ade-b0c8-9a3a73d4099f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:33:15.583273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-53942ee9-34c0-4ade-b0c8-9a3a73d4099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:15.713342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53942ee9-34c0-4ade-b0c8-9a3a73d4099f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:33:15.803477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee6196-9462-4253-8e51-dacf87b3400f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:33:15.804346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee6196-9462-4253-8e51-dacf87b3400f service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:33:15.804724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee6196-9462-4253-8e51-dacf87b3400f service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:33:15.805005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee6196-9462-4253-8e51-dacf87b3400f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:33:15.806887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c0ee6196-9462-4253-8e51-dacf87b3400f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:33:15.817110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:15.921522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee6196-9462-4253-8e51-dacf87b3400f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:33:15.922934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee6196-9462-4253-8e51-dacf87b3400f service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:33:15.941583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 743/1493] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:33:25.290557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=82 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:33:25.939270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=50 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:39.966705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82be7c-14fa-4df0-8cc4-ad0cde757197 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:39.967344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82be7c-14fa-4df0-8cc4-ad0cde757197 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:39.967344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82be7c-14fa-4df0-8cc4-ad0cde757197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:40.043801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8d82be7c-14fa-4df0-8cc4-ad0cde757197 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:34:40.045032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8d82be7c-14fa-4df0-8cc4-ad0cde757197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:40.142245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 751/1494] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:34:40.150598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f848d63-f8c9-49ea-bb7f-bd642a208f47 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:40.151107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f848d63-f8c9-49ea-bb7f-bd642a208f47 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:40.151223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f848d63-f8c9-49ea-bb7f-bd642a208f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:40.229550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7f848d63-f8c9-49ea-bb7f-bd642a208f47 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:34:40.230759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7f848d63-f8c9-49ea-bb7f-bd642a208f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:40.326696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 744/1495] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:34:40.331670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82be7c-14fa-4df0-8cc4-ad0cde757197 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:40.332905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8d82be7c-14fa-4df0-8cc4-ad0cde757197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:40.344773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82be7c-14fa-4df0-8cc4-ad0cde757197 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:40.412243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:40.446949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:40.447550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:40.447829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:40.448102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:40.448820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:34:40.449160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:40.450290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:34:40.450290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Aug 24 13:34:40.450290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: Aug 24 13:34:40.451504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:40.463635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 752/1496] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:34:40.471976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f848d63-f8c9-49ea-bb7f-bd642a208f47 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:40.482207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f848d63-f8c9-49ea-bb7f-bd642a208f47 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:40.543236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:40.577383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:40.577914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:40.578198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:40.578501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:40.579145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:40.580985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:34:40.580985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Aug 24 13:34:40.580985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: Aug 24 13:34:40.582181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:40.723534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 745/1497] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:34:40 2022] POST /identity/v3/projects => generated 379 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:34:40.728049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:40.729654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:40.765336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb909fc8-fd4a-4e3c-a4c6-bdd959597033 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:40.860660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:40.861182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:40.861458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:40.861757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:40.862369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:40.863370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:34:40.863370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Aug 24 13:34:40.863370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: Aug 24 13:34:40.864593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:40.969282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 753/1498] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:34:40 2022] POST /identity/v3/users => generated 290 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:34:40.972678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:40.974229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:41.049369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4baa9a-0635-447d-9afd-1c23b139f2cd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:41.134195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:41.144398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:41.144999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:41.145276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:41.145544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:41.146187 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:41.147308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:34:41.147308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Aug 24 13:34:41.147308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: Aug 24 13:34:41.148854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:41.161582 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 746/1499] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:34:41.165932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:41.167513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:41.179154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72b32d-ae59-4808-ac29-54f7fab15d34 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:41.205721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:41.279173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc92680-245a-4aa2-b5cc-d8d96109bb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:41.279780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc92680-245a-4aa2-b5cc-d8d96109bb8b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:41.280061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc92680-245a-4aa2-b5cc-d8d96109bb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:41.280375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc92680-245a-4aa2-b5cc-d8d96109bb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:41.281027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc92680-245a-4aa2-b5cc-d8d96109bb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:41.282047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-4bc92680-245a-4aa2-b5cc-d8d96109bb8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:34:41.282047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Aug 24 13:34:41.282047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: Aug 24 13:34:41.282946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc92680-245a-4aa2-b5cc-d8d96109bb8b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:41.296047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 754/1500] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:34:41.300276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:41.301840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:41.337525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc9f16-67f5-4bf2-a227-a97d2fff133f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:41.432641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdb8ef-f6ea-4203-a5cf-e4acd4ff55dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:41.433201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdb8ef-f6ea-4203-a5cf-e4acd4ff55dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:41.433477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdb8ef-f6ea-4203-a5cf-e4acd4ff55dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:41.433749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdb8ef-f6ea-4203-a5cf-e4acd4ff55dc admin admin] PATH_INFO: `/v3/projects/4e4adc81d769417789ac0f1b7889d2e4/users/25457333ce464f359c6ffcd8bbf17f6d/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:41.455824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdb8ef-f6ea-4203-a5cf-e4acd4ff55dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4adc81d769417789ac0f1b7889d2e4, user_id=25457333ce464f359c6ffcd8bbf17f6d, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:41.456895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-0cfdb8ef-f6ea-4203-a5cf-e4acd4ff55dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:34:41.456895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Aug 24 13:34:41.456895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: Aug 24 13:34:41.458824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdb8ef-f6ea-4203-a5cf-e4acd4ff55dc admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:41.564354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 747/1501] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:34:41 2022] PUT /identity/v3/projects/4e4adc81d769417789ac0f1b7889d2e4/users/25457333ce464f359c6ffcd8bbf17f6d/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:34:41.577473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24441100-fe82-40a1-a2d5-b0a15766964d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:41.577767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24441100-fe82-40a1-a2d5-b0a15766964d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:41.578049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24441100-fe82-40a1-a2d5-b0a15766964d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:41.638047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-24441100-fe82-40a1-a2d5-b0a15766964d None None] MFA Rules not processed for user `25457333ce464f359c6ffcd8bbf17f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:34:41.639251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-24441100-fe82-40a1-a2d5-b0a15766964d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:41.719163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 755/1502] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:34:41.732693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f0836-2292-42a8-ab7f-e93f837ce426 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:41.732984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f0836-2292-42a8-ab7f-e93f837ce426 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:41.733263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f0836-2292-42a8-ab7f-e93f837ce426 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:41.734603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 748/1503] 10.222.0.73 () {54 vars in 964 bytes} [Wed Aug 24 13:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:34:41.739247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f619a8-1cdf-4de1-ac87-09b0ae68ed74 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:41.739564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f619a8-1cdf-4de1-ac87-09b0ae68ed74 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:41.739842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f619a8-1cdf-4de1-ac87-09b0ae68ed74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:41.772291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:41.817584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-41f619a8-1cdf-4de1-ac87-09b0ae68ed74 None None] MFA Rules not processed for user `2550e85fa53d416887bff88fbff937d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:34:41.819502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41f619a8-1cdf-4de1-ac87-09b0ae68ed74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:41.903578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 756/1504] 10.222.0.73 () {58 vars in 1061 bytes} [Wed Aug 24 13:34:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:34:41.911722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f0836-2292-42a8-ab7f-e93f837ce426 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:41.913300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fc2f0836-2292-42a8-ab7f-e93f837ce426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:41.964335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:42.020520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f0836-2292-42a8-ab7f-e93f837ce426 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:42.096400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:42.097008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:42.097294 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:42.097570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:42.099195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:42.202547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:42.203994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:42.207600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:42.222203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 749/1505] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:34:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:34:42.268804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f638b-4038-4281-98b5-2c5df1988a09 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:42.269208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f638b-4038-4281-98b5-2c5df1988a09 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:42.269357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f638b-4038-4281-98b5-2c5df1988a09 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:42.270694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 757/1506] 10.222.0.73 () {52 vars in 830 bytes} [Wed Aug 24 13:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:34:42.274247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:42.275467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:42.377107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2786f-d090-486c-944e-dd5364e53191 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:42.472398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:42.473032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:42.473314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:42.473589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] PATH_INFO: `/v3/users/25457333ce464f359c6ffcd8bbf17f6d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:42.486601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25457333ce464f359c6ffcd8bbf17f6d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:42.487647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:34:42.487647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Aug 24 13:34:42.487647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: Aug 24 13:34:42.488905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:42.498820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:42.669849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25457333ce464f359c6ffcd8bbf17f6d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:42.670209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25457333ce464f359c6ffcd8bbf17f6d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:42.685491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25457333ce464f359c6ffcd8bbf17f6d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:42.686054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25457333ce464f359c6ffcd8bbf17f6d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:42.749853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25457333ce464f359c6ffcd8bbf17f6d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:42.780432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 750/1507] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:34:42 2022] DELETE /identity/v3/users/25457333ce464f359c6ffcd8bbf17f6d => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:34:42.784852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa3dba-fe16-4729-9503-94a204c1ff9f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:42.785146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa3dba-fe16-4729-9503-94a204c1ff9f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:42.785427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa3dba-fe16-4729-9503-94a204c1ff9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:42.846214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f3fa3dba-fe16-4729-9503-94a204c1ff9f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:34:42.847442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f3fa3dba-fe16-4729-9503-94a204c1ff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:42.923311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:42.936307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 758/1508] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:34:42.946557 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:42.947795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:43.056761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c59058-1237-4aac-926a-6a8cce8ce5d3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:43.112767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:43.131213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee871f8-4030-4070-bb63-16def2ca528b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:43.131780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee871f8-4030-4070-bb63-16def2ca528b service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:43.132058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee871f8-4030-4070-bb63-16def2ca528b service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:43.132362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee871f8-4030-4070-bb63-16def2ca528b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:43.134594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dee871f8-4030-4070-bb63-16def2ca528b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:43.368659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee871f8-4030-4070-bb63-16def2ca528b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:43.370203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee871f8-4030-4070-bb63-16def2ca528b service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:43.390077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 751/1509] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:34:43.878554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa3dba-fe16-4729-9503-94a204c1ff9f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:34:43.879787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f3fa3dba-fe16-4729-9503-94a204c1ff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:34:43.895121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa3dba-fe16-4729-9503-94a204c1ff9f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:34:44.000285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:34:44.000919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:34:44.001198 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:34:44.001467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] PATH_INFO: `/v3/projects/4e4adc81d769417789ac0f1b7889d2e4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:34:44.012453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4adc81d769417789ac0f1b7889d2e4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:34:44.013476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:34:44.013476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Aug 24 13:34:44.013476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: Aug 24 13:34:44.014672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:34:44.143455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4adc81d769417789ac0f1b7889d2e4', 'request_id': 'req-fd85de5f-215f-4eb3-96cf-f5c598529e3b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:44.150170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4adc81d769417789ac0f1b7889d2e4', 'request_id': 'req-fd85de5f-215f-4eb3-96cf-f5c598529e3b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:44.195433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4adc81d769417789ac0f1b7889d2e4', 'request_id': 'req-fd85de5f-215f-4eb3-96cf-f5c598529e3b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:44.229008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] The token cache is being invalidate because project 4e4adc81d769417789ac0f1b7889d2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:34:44.229495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:44.229831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-fd85de5f-215f-4eb3-96cf-f5c598529e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:34:44.231168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 759/1510] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:34:43 2022] DELETE /identity/v3/projects/4e4adc81d769417789ac0f1b7889d2e4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:34:53.387589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=62 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:34:54.226044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:36:10.981537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc01dd5-d90a-48bc-bfd1-70e2fa88b2ce None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:10.982420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc01dd5-d90a-48bc-bfd1-70e2fa88b2ce None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:10.982420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc01dd5-d90a-48bc-bfd1-70e2fa88b2ce None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:10.983517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:36:10.991066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf317e-bb97-48b0-a0d7-311f0c4912f3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:10.991670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf317e-bb97-48b0-a0d7-311f0c4912f3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:10.992103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf317e-bb97-48b0-a0d7-311f0c4912f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:11.149335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-90bf317e-bb97-48b0-a0d7-311f0c4912f3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:11.151313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-90bf317e-bb97-48b0-a0d7-311f0c4912f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:11.306549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:36:11.470537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc01dd5-d90a-48bc-bfd1-70e2fa88b2ce None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:11.480409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fdc01dd5-d90a-48bc-bfd1-70e2fa88b2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:11.621146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc01dd5-d90a-48bc-bfd1-70e2fa88b2ce None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:11.627491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097eb1c2-ab2d-4977-af1d-44d9323026d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:11.628143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eb1c2-ab2d-4977-af1d-44d9323026d5 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:11.628586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eb1c2-ab2d-4977-af1d-44d9323026d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:11.628859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097eb1c2-ab2d-4977-af1d-44d9323026d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:11.639521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097eb1c2-ab2d-4977-af1d-44d9323026d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:11.641254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097eb1c2-ab2d-4977-af1d-44d9323026d5 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:11.653582 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 753/1513] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:36:12.636817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407bf7b-b6fe-4e14-9133-b93b0d86032f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:12.637719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407bf7b-b6fe-4e14-9133-b93b0d86032f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:12.637719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407bf7b-b6fe-4e14-9133-b93b0d86032f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:12.639419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:36:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:36:12.645488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f3c5b-1644-48ad-97fb-96c228ec51a1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:12.646776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f3c5b-1644-48ad-97fb-96c228ec51a1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:12.646947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f3c5b-1644-48ad-97fb-96c228ec51a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:12.669922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-321f3c5b-1644-48ad-97fb-96c228ec51a1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:12.671201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-321f3c5b-1644-48ad-97fb-96c228ec51a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:12.680100 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:36:12.859014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407bf7b-b6fe-4e14-9133-b93b0d86032f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:12.869452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a06b7-f659-457f-88cb-ffa485188d98 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:12.869900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a06b7-f659-457f-88cb-ffa485188d98 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:12.870361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a06b7-f659-457f-88cb-ffa485188d98 None None] PATH_INFO: `/v3` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:12.872820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 755/1516] 10.222.0.20 () {54 vars in 925 bytes} [Wed Aug 24 13:36:12 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:36:12.880310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31379164-e0dc-479c-90a0-f12907e5b72d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:12.880620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407bf7b-b6fe-4e14-9133-b93b0d86032f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:12.881025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31379164-e0dc-479c-90a0-f12907e5b72d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:12.881519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31379164-e0dc-479c-90a0-f12907e5b72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:12.888654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a7ac6-b521-4f46-8f77-396d5278d214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:12.889439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a7ac6-b521-4f46-8f77-396d5278d214 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:12.889851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a7ac6-b521-4f46-8f77-396d5278d214 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:12.890257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a7ac6-b521-4f46-8f77-396d5278d214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:12.902567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a7ac6-b521-4f46-8f77-396d5278d214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:12.912036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a7ac6-b521-4f46-8f77-396d5278d214 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:12.919417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 762/1517] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:36:12.965435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-31379164-e0dc-479c-90a0-f12907e5b72d None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:12.966277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-31379164-e0dc-479c-90a0-f12907e5b72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:13.061315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:36:13.087410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 756/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 24 13:36:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:36:22.595496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42614569-7a92-427a-b09c-8179fc6e505b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:22.596071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42614569-7a92-427a-b09c-8179fc6e505b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:22.596071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42614569-7a92-427a-b09c-8179fc6e505b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:22.597425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 763/1519] 10.222.0.73 () {54 vars in 961 bytes} [Wed Aug 24 13:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:36:22.603905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ab7f0-ef29-4a09-8f9c-2a4eec1c54d0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:22.604751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ab7f0-ef29-4a09-8f9c-2a4eec1c54d0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:22.604751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ab7f0-ef29-4a09-8f9c-2a4eec1c54d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:22.630887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-570ab7f0-ef29-4a09-8f9c-2a4eec1c54d0 None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:22.632143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-570ab7f0-ef29-4a09-8f9c-2a4eec1c54d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:22.642395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 757/1520] 10.222.0.73 () {58 vars in 1058 bytes} [Wed Aug 24 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:36:22.653746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42614569-7a92-427a-b09c-8179fc6e505b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:22.668640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42614569-7a92-427a-b09c-8179fc6e505b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:22.673718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181b68a-667c-4d72-8d66-f880158a72bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:22.674418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181b68a-667c-4d72-8d66-f880158a72bb service placement] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:22.674613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181b68a-667c-4d72-8d66-f880158a72bb service placement] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:22.674888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181b68a-667c-4d72-8d66-f880158a72bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:22.685688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181b68a-667c-4d72-8d66-f880158a72bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:22.687299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181b68a-667c-4d72-8d66-f880158a72bb service placement] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:22.697953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 764/1521] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:36:24.240000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7ad4f-b8fe-4af1-a6a1-48393d112433 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:24.240610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7ad4f-b8fe-4af1-a6a1-48393d112433 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:24.240610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7ad4f-b8fe-4af1-a6a1-48393d112433 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:24.241924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:36:24.249628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee9fd-9fa2-41a0-8146-77c000aaa9c7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:24.250138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee9fd-9fa2-41a0-8146-77c000aaa9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:24.250232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ee9fd-9fa2-41a0-8146-77c000aaa9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:24.273180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1b2ee9fd-9fa2-41a0-8146-77c000aaa9c7 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:24.274442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1b2ee9fd-9fa2-41a0-8146-77c000aaa9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:24.282652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:36:24.447290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7ad4f-b8fe-4af1-a6a1-48393d112433 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:24.464537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7ad4f-b8fe-4af1-a6a1-48393d112433 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:24.470251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06483c-6221-46dc-ba22-23be699f87e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:24.470871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06483c-6221-46dc-ba22-23be699f87e6 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:24.471153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06483c-6221-46dc-ba22-23be699f87e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:24.471432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06483c-6221-46dc-ba22-23be699f87e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:24.481430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06483c-6221-46dc-ba22-23be699f87e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:24.483065 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06483c-6221-46dc-ba22-23be699f87e6 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:24.494016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 759/1524] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:36:25.370966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a13a11-04bc-4732-9e39-dd12b89350e6 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:25.371564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a13a11-04bc-4732-9e39-dd12b89350e6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:25.371564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a13a11-04bc-4732-9e39-dd12b89350e6 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:25.372956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 13:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:36:25.380949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724a64-6b06-4575-99ae-54584f0bf5d4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:25.380949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724a64-6b06-4575-99ae-54584f0bf5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:25.380949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724a64-6b06-4575-99ae-54584f0bf5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:25.399316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1e724a64-6b06-4575-99ae-54584f0bf5d4 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:25.400568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1e724a64-6b06-4575-99ae-54584f0bf5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:25.410095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 13:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:36:25.579217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a13a11-04bc-4732-9e39-dd12b89350e6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:25.597259 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a13a11-04bc-4732-9e39-dd12b89350e6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:25.605095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa0926-479a-4571-8d2e-687aaa5fe80f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:25.605838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa0926-479a-4571-8d2e-687aaa5fe80f service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:25.606214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa0926-479a-4571-8d2e-687aaa5fe80f service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:25.606590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa0926-479a-4571-8d2e-687aaa5fe80f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:25.617193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa0926-479a-4571-8d2e-687aaa5fe80f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:25.618810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa0926-479a-4571-8d2e-687aaa5fe80f service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:25.628861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 767/1527] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:36:29.720468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c779bc-5d8b-4575-a2ef-38eecec1cf75 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:29.721050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c779bc-5d8b-4575-a2ef-38eecec1cf75 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:29.721050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c779bc-5d8b-4575-a2ef-38eecec1cf75 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:29.722436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:36:29.728745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815026e-462a-47b7-9462-5944d8137b58 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:29.729295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815026e-462a-47b7-9462-5944d8137b58 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:29.729557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815026e-462a-47b7-9462-5944d8137b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:29.752745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2815026e-462a-47b7-9462-5944d8137b58 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:29.753979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2815026e-462a-47b7-9462-5944d8137b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:29.762738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:36:30.055846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c779bc-5d8b-4575-a2ef-38eecec1cf75 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:30.057821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d9c779bc-5d8b-4575-a2ef-38eecec1cf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:30.148804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c779bc-5d8b-4575-a2ef-38eecec1cf75 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:30.152514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9727a1-5bb7-4b3f-9590-6752532de201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:30.153130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9727a1-5bb7-4b3f-9590-6752532de201 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:30.153411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9727a1-5bb7-4b3f-9590-6752532de201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:30.153684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9727a1-5bb7-4b3f-9590-6752532de201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:30.163988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9727a1-5bb7-4b3f-9590-6752532de201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:30.165467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9727a1-5bb7-4b3f-9590-6752532de201 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:30.175671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 762/1530] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:36:32.188976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838f156-8081-4a89-8988-925aed37769b None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:32.189918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838f156-8081-4a89-8988-925aed37769b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:32.189918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838f156-8081-4a89-8988-925aed37769b None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:32.191488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:36:32.197675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beed07b-f516-4614-87f4-e729c7ea72c1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:32.198352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beed07b-f516-4614-87f4-e729c7ea72c1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:32.198352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beed07b-f516-4614-87f4-e729c7ea72c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:32.219944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2beed07b-f516-4614-87f4-e729c7ea72c1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:32.221243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2beed07b-f516-4614-87f4-e729c7ea72c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:32.230193 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:36:32.522752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838f156-8081-4a89-8988-925aed37769b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:32.534488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838f156-8081-4a89-8988-925aed37769b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:32.540165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f4c8b-a9d4-499b-a0aa-cf5e7cbb3451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:32.541162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4c8b-a9d4-499b-a0aa-cf5e7cbb3451 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:32.541453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4c8b-a9d4-499b-a0aa-cf5e7cbb3451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:32.541817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4c8b-a9d4-499b-a0aa-cf5e7cbb3451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:32.552165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f4c8b-a9d4-499b-a0aa-cf5e7cbb3451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:32.553646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f4c8b-a9d4-499b-a0aa-cf5e7cbb3451 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:32.565148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 770/1533] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:36:36.317828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa0d0-76a1-4c83-97f0-25610487cc2b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:36.318505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa0d0-76a1-4c83-97f0-25610487cc2b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:36.318505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa0d0-76a1-4c83-97f0-25610487cc2b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:36.320501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:36:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:36:36.326368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dc5f2-d12b-40d7-a05c-fe284ca9ef1d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:36.326924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dc5f2-d12b-40d7-a05c-fe284ca9ef1d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:36.327050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dc5f2-d12b-40d7-a05c-fe284ca9ef1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:36.335609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:36:36.348557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-839dc5f2-d12b-40d7-a05c-fe284ca9ef1d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:36.350226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-839dc5f2-d12b-40d7-a05c-fe284ca9ef1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:36.362415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:36:36.936011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaa0d0-76a1-4c83-97f0-25610487cc2b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:36.938197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-91eaa0d0-76a1-4c83-97f0-25610487cc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:37.044673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaa0d0-76a1-4c83-97f0-25610487cc2b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:37.048936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f367c1-7369-4e71-8e64-209daae64872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:37.049562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f367c1-7369-4e71-8e64-209daae64872 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:37.049841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f367c1-7369-4e71-8e64-209daae64872 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:37.050116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f367c1-7369-4e71-8e64-209daae64872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:37.061859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f367c1-7369-4e71-8e64-209daae64872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:37.063381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f367c1-7369-4e71-8e64-209daae64872 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:37.075379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 765/1536] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:36:39.165744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021e8d1-dc05-4cd7-9ffe-94329ae89f63 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:39.166681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021e8d1-dc05-4cd7-9ffe-94329ae89f63 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:39.166681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021e8d1-dc05-4cd7-9ffe-94329ae89f63 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:39.169391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 13:36:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:36:39.175227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faf87a-1062-4c21-992d-d374bb0288f3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:39.175939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faf87a-1062-4c21-992d-d374bb0288f3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:39.175939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faf87a-1062-4c21-992d-d374bb0288f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:39.199348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e4faf87a-1062-4c21-992d-d374bb0288f3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:36:39.200613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e4faf87a-1062-4c21-992d-d374bb0288f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:39.209548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 13:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:36:39.786164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021e8d1-dc05-4cd7-9ffe-94329ae89f63 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:36:39.788355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1021e8d1-dc05-4cd7-9ffe-94329ae89f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:36:39.802754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021e8d1-dc05-4cd7-9ffe-94329ae89f63 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:36:39.806603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b0c11-79b7-444a-a77c-155c3c31c20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:36:39.807498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b0c11-79b7-444a-a77c-155c3c31c20f service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:36:39.808171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b0c11-79b7-444a-a77c-155c3c31c20f service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:36:39.808478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b0c11-79b7-444a-a77c-155c3c31c20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:36:39.820296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b0c11-79b7-444a-a77c-155c3c31c20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:36:39.821780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b0c11-79b7-444a-a77c-155c3c31c20f service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:36:39.835559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 773/1539] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:36:49.194226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:36:49.827871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=8 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:05.945640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645058f4-986d-4698-b406-6e86eda95e7d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:05.946740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645058f4-986d-4698-b406-6e86eda95e7d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:05.946740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645058f4-986d-4698-b406-6e86eda95e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:05.971353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4ed5-048d-430e-8bce-1dcdfe70b1b0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:05.972312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4ed5-048d-430e-8bce-1dcdfe70b1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:05.972815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb4ed5-048d-430e-8bce-1dcdfe70b1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.039312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-645058f4-986d-4698-b406-6e86eda95e7d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:06.042049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-645058f4-986d-4698-b406-6e86eda95e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:06.049878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4afb4ed5-048d-430e-8bce-1dcdfe70b1b0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:06.051974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4afb4ed5-048d-430e-8bce-1dcdfe70b1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:06.087468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 767/1540] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:06.097693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290983d-2bb2-4b6d-9baa-8d4879b5118e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.098268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290983d-2bb2-4b6d-9baa-8d4879b5118e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.098767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290983d-2bb2-4b6d-9baa-8d4879b5118e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.105064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 774/1541] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:06.116821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73101c61-0ac5-4e79-83b7-32f03f4cb686 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.117358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73101c61-0ac5-4e79-83b7-32f03f4cb686 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.117823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73101c61-0ac5-4e79-83b7-32f03f4cb686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.173246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-73101c61-0ac5-4e79-83b7-32f03f4cb686 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:06.173613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2290983d-2bb2-4b6d-9baa-8d4879b5118e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:06.174968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-73101c61-0ac5-4e79-83b7-32f03f4cb686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:06.175188 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2290983d-2bb2-4b6d-9baa-8d4879b5118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:06.196385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 768/1542] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:06.198767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 775/1543] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:06.205450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73101c61-0ac5-4e79-83b7-32f03f4cb686 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:06.205987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290983d-2bb2-4b6d-9baa-8d4879b5118e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:06.227991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290983d-2bb2-4b6d-9baa-8d4879b5118e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:06.231862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73101c61-0ac5-4e79-83b7-32f03f4cb686 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:06.238324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:06.238968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.239263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.239551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.240339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:06.240653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:06.241715 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:38:06.241715 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Aug 24 13:38:06.241715 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: Aug 24 13:38:06.249228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:06.250463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:06.251392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.251724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.252016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.257059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:06.257529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:06.259928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:06.279296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 769/1544] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:06.288086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:06.300005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 776/1545] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:38:06.310025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:06.311210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc37a0-e02a-46e6-9f0e-c7d5c849da82 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:06.320928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f581b-9066-4bbf-9421-37501215c492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:06.321566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f581b-9066-4bbf-9421-37501215c492 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.322901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f581b-9066-4bbf-9421-37501215c492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.322901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f581b-9066-4bbf-9421-37501215c492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.322901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f581b-9066-4bbf-9421-37501215c492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:06.325248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f581b-9066-4bbf-9421-37501215c492 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:06.333244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02100e-20ed-4f7a-afdd-3c874f4ba3db admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:06.350002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96d5ec-ef4f-49fd-8c7f-534f320d55e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:06.350625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d5ec-ef4f-49fd-8c7f-534f320d55e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.351093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d5ec-ef4f-49fd-8c7f-534f320d55e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.351383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96d5ec-ef4f-49fd-8c7f-534f320d55e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.352032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96d5ec-ef4f-49fd-8c7f-534f320d55e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:06.353220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-ad96d5ec-ef4f-49fd-8c7f-534f320d55e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:38:06.353220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Aug 24 13:38:06.353220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: Aug 24 13:38:06.354429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96d5ec-ef4f-49fd-8c7f-534f320d55e8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:06.520384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 770/1546] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:06 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:38:06.524438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f581b-9066-4bbf-9421-37501215c492 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:06.570376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 777/1547] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:06 2022] POST /identity/v3/projects => generated 373 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:06.571702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c6bc0-45b8-449b-b34e-ad5652e86af4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.572000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c6bc0-45b8-449b-b34e-ad5652e86af4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.572637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c6bc0-45b8-449b-b34e-ad5652e86af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.606437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-199c6bc0-45b8-449b-b34e-ad5652e86af4 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:06.609009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-199c6bc0-45b8-449b-b34e-ad5652e86af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:06.650946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f581b-9066-4bbf-9421-37501215c492 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:06.815940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 778/1548] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:06.816917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c6bc0-45b8-449b-b34e-ad5652e86af4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:06.826369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ab264-0aee-4870-9c06-dfea89df20e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:06.826916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab264-0aee-4870-9c06-dfea89df20e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.827391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab264-0aee-4870-9c06-dfea89df20e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.827829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ab264-0aee-4870-9c06-dfea89df20e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.828137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab264-0aee-4870-9c06-dfea89df20e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:06.832527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-a10ab264-0aee-4870-9c06-dfea89df20e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:38:06.832527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Aug 24 13:38:06.832527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: Aug 24 13:38:06.832527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ab264-0aee-4870-9c06-dfea89df20e4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:06.846887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c6bc0-45b8-449b-b34e-ad5652e86af4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:06.859773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b068264-244e-4bfa-ba59-f203b635bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:06.862765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b068264-244e-4bfa-ba59-f203b635bec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.863085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b068264-244e-4bfa-ba59-f203b635bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.863368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b068264-244e-4bfa-ba59-f203b635bec8 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:06.864047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b068264-244e-4bfa-ba59-f203b635bec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:06.866564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b068264-244e-4bfa-ba59-f203b635bec8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:06.975163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 771/1549] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:06 2022] POST /identity/v3/users => generated 293 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:38:06.977691 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caf092-117e-4fe8-aa5c-e19a5ce991cd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:06.979522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caf092-117e-4fe8-aa5c-e19a5ce991cd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:06.980390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caf092-117e-4fe8-aa5c-e19a5ce991cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.024145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-92caf092-117e-4fe8-aa5c-e19a5ce991cd None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.026597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-92caf092-117e-4fe8-aa5c-e19a5ce991cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.034678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 779/1550] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:06 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:07.041116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ebe7e-f1b0-4c13-99fa-6bf7bb2cafad None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.042454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ebe7e-f1b0-4c13-99fa-6bf7bb2cafad None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.042454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ebe7e-f1b0-4c13-99fa-6bf7bb2cafad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.049459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 772/1551] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.054646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64170ca1-d897-41a0-868c-19c75e5bb569 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.056404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64170ca1-d897-41a0-868c-19c75e5bb569 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.058526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64170ca1-d897-41a0-868c-19c75e5bb569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.071124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e09ebe7e-f1b0-4c13-99fa-6bf7bb2cafad None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.074474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e09ebe7e-f1b0-4c13-99fa-6bf7bb2cafad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.083181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 780/1552] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.084714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2856a11-9c07-4464-91e9-d0ccc835f4d9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.084857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2856a11-9c07-4464-91e9-d0ccc835f4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.085048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2856a11-9c07-4464-91e9-d0ccc835f4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.093816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-64170ca1-d897-41a0-868c-19c75e5bb569 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.096685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-64170ca1-d897-41a0-868c-19c75e5bb569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.109049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b2856a11-9c07-4464-91e9-d0ccc835f4d9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.111955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 773/1553] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.112209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b2856a11-9c07-4464-91e9-d0ccc835f4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.116960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64170ca1-d897-41a0-868c-19c75e5bb569 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.126526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 781/1554] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.128664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703aa4c-11e1-45c3-a940-fd9be4f0ef42 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.128918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64170ca1-d897-41a0-868c-19c75e5bb569 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.129161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703aa4c-11e1-45c3-a940-fd9be4f0ef42 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.129631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703aa4c-11e1-45c3-a940-fd9be4f0ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.136702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0cbc0-3285-4a2d-b25f-0cf7f1f3376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.137489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0cbc0-3285-4a2d-b25f-0cf7f1f3376b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.137865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0cbc0-3285-4a2d-b25f-0cf7f1f3376b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.138235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0cbc0-3285-4a2d-b25f-0cf7f1f3376b admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.139118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0cbc0-3285-4a2d-b25f-0cf7f1f3376b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.140894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0cbc0-3285-4a2d-b25f-0cf7f1f3376b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.152892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 774/1555] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:07.153939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0cbc0-3285-4a2d-b25f-0cf7f1f3376b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.160976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6703aa4c-11e1-45c3-a940-fd9be4f0ef42 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.162905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6703aa4c-11e1-45c3-a940-fd9be4f0ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.167468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0cbc0-3285-4a2d-b25f-0cf7f1f3376b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.172546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52180f67-bc66-4fad-bfd5-9407f7e340ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.172997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 782/1556] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.173187 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52180f67-bc66-4fad-bfd5-9407f7e340ee admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.174138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52180f67-bc66-4fad-bfd5-9407f7e340ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.174478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52180f67-bc66-4fad-bfd5-9407f7e340ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.175180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52180f67-bc66-4fad-bfd5-9407f7e340ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.176130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb6282-d51b-43f9-b48a-3cd8ef3c7132 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.176605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb6282-d51b-43f9-b48a-3cd8ef3c7132 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.176842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52180f67-bc66-4fad-bfd5-9407f7e340ee admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.177098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb6282-d51b-43f9-b48a-3cd8ef3c7132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.192155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 775/1557] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:07.193577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55092ad-b5fc-4551-bce3-b0a4aa129c36 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.193934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55092ad-b5fc-4551-bce3-b0a4aa129c36 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.194274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55092ad-b5fc-4551-bce3-b0a4aa129c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.206792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:07.208670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-abcb6282-d51b-43f9-b48a-3cd8ef3c7132 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.212539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-abcb6282-d51b-43f9-b48a-3cd8ef3c7132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.225133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 783/1558] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.226276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e55092ad-b5fc-4551-bce3-b0a4aa129c36 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.226950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256acc3-3c99-4cb3-a372-e84e0e61bd61 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.227409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256acc3-3c99-4cb3-a372-e84e0e61bd61 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.227870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256acc3-3c99-4cb3-a372-e84e0e61bd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.228279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e55092ad-b5fc-4551-bce3-b0a4aa129c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.243518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 776/1559] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.245093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55092ad-b5fc-4551-bce3-b0a4aa129c36 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.258035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b256acc3-3c99-4cb3-a372-e84e0e61bd61 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.259331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b256acc3-3c99-4cb3-a372-e84e0e61bd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.261788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55092ad-b5fc-4551-bce3-b0a4aa129c36 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.267202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.288725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 784/1560] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.288969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.288969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.289245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda1c3-c96f-4813-bcc8-b57037d7200b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.289401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.289547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda1c3-c96f-4813-bcc8-b57037d7200b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.289547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda1c3-c96f-4813-bcc8-b57037d7200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.289843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:07.289843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.289843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.291293 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 777/1561] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:07.292223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-40cda1c3-c96f-4813-bcc8-b57037d7200b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.292620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.293488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-40cda1c3-c96f-4813-bcc8-b57037d7200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.307989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 785/1562] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.309854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9de9-3381-40fa-95f5-6eb0bba6b6f2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.310330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9de9-3381-40fa-95f5-6eb0bba6b6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.310836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc9de9-3381-40fa-95f5-6eb0bba6b6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.316790 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d98a7e-5b0a-4a49-953e-c73a92997bb3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.322778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c78252-a9a8-4e7a-918b-e7ef42fb7fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.323409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c78252-a9a8-4e7a-918b-e7ef42fb7fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.323728 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c78252-a9a8-4e7a-918b-e7ef42fb7fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.324066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c78252-a9a8-4e7a-918b-e7ef42fb7fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.324756 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c78252-a9a8-4e7a-918b-e7ef42fb7fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.326512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c78252-a9a8-4e7a-918b-e7ef42fb7fbe admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.335435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-75cc9de9-3381-40fa-95f5-6eb0bba6b6f2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.336622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-75cc9de9-3381-40fa-95f5-6eb0bba6b6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.340899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 778/1563] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:07.341683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c78252-a9a8-4e7a-918b-e7ef42fb7fbe admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.347654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 786/1564] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.349065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b584a-7215-49b4-8e2f-f50655854db5 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.349347 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b584a-7215-49b4-8e2f-f50655854db5 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.349649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b584a-7215-49b4-8e2f-f50655854db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.363027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c78252-a9a8-4e7a-918b-e7ef42fb7fbe admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.370529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12cc95-1aba-4a76-a518-eafe5a9f2740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.371411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12cc95-1aba-4a76-a518-eafe5a9f2740 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.371873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12cc95-1aba-4a76-a518-eafe5a9f2740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.372328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12cc95-1aba-4a76-a518-eafe5a9f2740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.373448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12cc95-1aba-4a76-a518-eafe5a9f2740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.376225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12cc95-1aba-4a76-a518-eafe5a9f2740 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.381054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f20b584a-7215-49b4-8e2f-f50655854db5 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:07.382988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f20b584a-7215-49b4-8e2f-f50655854db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:07.395688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 779/1565] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:07.397171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12cc95-1aba-4a76-a518-eafe5a9f2740 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.400479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 787/1566] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:07.401912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b584a-7215-49b4-8e2f-f50655854db5 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.411817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12cc95-1aba-4a76-a518-eafe5a9f2740 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.413327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b584a-7215-49b4-8e2f-f50655854db5 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.418934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.419311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.419576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.419787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.419956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.420124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.420351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.420738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.421074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:07.421459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.421677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:07.422085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.423524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.424279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.442821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 788/1567] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:07.443268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 780/1568] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:07.444113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.444707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.461271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5626cf-402a-4e7e-a336-4f4826a57c03 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.461864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f370b09-bf51-44e6-a632-04c789fc7d17 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.466475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c7737-e6aa-4404-ae71-3380d2d27bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.466660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.467088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7737-e6aa-4404-ae71-3380d2d27bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.467461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7737-e6aa-4404-ae71-3380d2d27bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.467682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.467837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7737-e6aa-4404-ae71-3380d2d27bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.468034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.468284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c7737-e6aa-4404-ae71-3380d2d27bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.468552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.469314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:07.469716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.470203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c7737-e6aa-4404-ae71-3380d2d27bb3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.471829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.484200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 781/1569] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:07.486559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.496858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5a86e-95e5-4d48-afb8-a48b30e59aa5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.502237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc65a0-9612-48fb-934a-9a4bd19ee52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.503273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc65a0-9612-48fb-934a-9a4bd19ee52a admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.503563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc65a0-9612-48fb-934a-9a4bd19ee52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.503839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc65a0-9612-48fb-934a-9a4bd19ee52a admin admin] PATH_INFO: `/v3/projects/867d7c32e6b545f49079bfcb6886db3a/users/fce6a5fc556c4cec9fbe79960cce25ae/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.520466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc65a0-9612-48fb-934a-9a4bd19ee52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=867d7c32e6b545f49079bfcb6886db3a, user_id=fce6a5fc556c4cec9fbe79960cce25ae, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.523173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc65a0-9612-48fb-934a-9a4bd19ee52a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.593321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:07.601300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 789/1570] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/projects => generated 389 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:07.602184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c7737-e6aa-4404-ae71-3380d2d27bb3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.653176 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 782/1571] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:07 2022] PUT /identity/v3/projects/867d7c32e6b545f49079bfcb6886db3a/users/fce6a5fc556c4cec9fbe79960cce25ae/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:07.654711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc65a0-9612-48fb-934a-9a4bd19ee52a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.699294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c7737-e6aa-4404-ae71-3380d2d27bb3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.746995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc65a0-9612-48fb-934a-9a4bd19ee52a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.756115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecf187-2478-450e-8f01-a0086a194000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.757323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecf187-2478-450e-8f01-a0086a194000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.757814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecf187-2478-450e-8f01-a0086a194000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.758285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecf187-2478-450e-8f01-a0086a194000 admin admin] PATH_INFO: `/v3/projects/088b026900964e37972e536090575868/users/410cef275ee74b18afdbce7439e3ed41/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.776115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.776774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.777058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.777333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.777986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:07.778285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.779909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.787360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecf187-2478-450e-8f01-a0086a194000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=088b026900964e37972e536090575868, user_id=410cef275ee74b18afdbce7439e3ed41, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.790933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecf187-2478-450e-8f01-a0086a194000 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.793690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 790/1572] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:07.797218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.810356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac3554-7065-49ca-88dc-70cfe6b6c994 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.816594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.817235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.817633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.817921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.818592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:07.818899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.820655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.837479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 791/1573] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:07.843753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.850586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1760cd0-8b3d-48f0-92d2-4dadc61a29a7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.855424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58c6aa-a778-4440-92ba-33794d08cf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.856041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58c6aa-a778-4440-92ba-33794d08cf25 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.856502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58c6aa-a778-4440-92ba-33794d08cf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.856774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58c6aa-a778-4440-92ba-33794d08cf25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.857416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58c6aa-a778-4440-92ba-33794d08cf25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.859350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58c6aa-a778-4440-92ba-33794d08cf25 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.871401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 783/1574] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:07 2022] PUT /identity/v3/projects/088b026900964e37972e536090575868/users/410cef275ee74b18afdbce7439e3ed41/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:07.872564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecf187-2478-450e-8f01-a0086a194000 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:07.937818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:07.944572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecf187-2478-450e-8f01-a0086a194000 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:07.949461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7c515-b3d8-4286-9195-d29a27da8803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:07.950086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7c515-b3d8-4286-9195-d29a27da8803 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:07.950365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7c515-b3d8-4286-9195-d29a27da8803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:07.950636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7c515-b3d8-4286-9195-d29a27da8803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:07.951277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7c515-b3d8-4286-9195-d29a27da8803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:07.953243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7c515-b3d8-4286-9195-d29a27da8803 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:07.983802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 792/1575] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:07.983802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58c6aa-a778-4440-92ba-33794d08cf25 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.067306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 784/1576] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:08.068469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7c515-b3d8-4286-9195-d29a27da8803 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.071633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58c6aa-a778-4440-92ba-33794d08cf25 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.085487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:08.161225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38805914-87da-47e7-9d63-fb46abf714a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.161943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38805914-87da-47e7-9d63-fb46abf714a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.162463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38805914-87da-47e7-9d63-fb46abf714a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.162744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38805914-87da-47e7-9d63-fb46abf714a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.163588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38805914-87da-47e7-9d63-fb46abf714a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.166302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38805914-87da-47e7-9d63-fb46abf714a6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:08.170920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7c515-b3d8-4286-9195-d29a27da8803 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.179542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e365a-7d88-43a4-920a-9236d35a3e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.180622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e365a-7d88-43a4-920a-9236d35a3e0a admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.180991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e365a-7d88-43a4-920a-9236d35a3e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.181294 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e365a-7d88-43a4-920a-9236d35a3e0a admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.181998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e365a-7d88-43a4-920a-9236d35a3e0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.186023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e365a-7d88-43a4-920a-9236d35a3e0a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:08.297851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 793/1577] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:07 2022] POST /identity/v3/projects => generated 373 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:08.300293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb185b-665e-42f1-9280-6283f929fde0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.300776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb185b-665e-42f1-9280-6283f929fde0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.301309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb185b-665e-42f1-9280-6283f929fde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.314920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 785/1578] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/users => generated 295 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:08.315846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e365a-7d88-43a4-920a-9236d35a3e0a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.335444 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e365a-7d88-43a4-920a-9236d35a3e0a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.337703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-74fb185b-665e-42f1-9280-6283f929fde0 None None] MFA Rules not processed for user `fce6a5fc556c4cec9fbe79960cce25ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:08.338957 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-74fb185b-665e-42f1-9280-6283f929fde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:08.416298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6039004-01c3-49ec-a8d1-ae10da872e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.416973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039004-01c3-49ec-a8d1-ae10da872e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.417384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039004-01c3-49ec-a8d1-ae10da872e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.417678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039004-01c3-49ec-a8d1-ae10da872e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.418341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6039004-01c3-49ec-a8d1-ae10da872e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.418793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 794/1579] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:08.419708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb185b-665e-42f1-9280-6283f929fde0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.420482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6039004-01c3-49ec-a8d1-ae10da872e3d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:08.436962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb185b-665e-42f1-9280-6283f929fde0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.445226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3b949-bb42-4709-a0e8-2da4214e6135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.446219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b949-bb42-4709-a0e8-2da4214e6135 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.446636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b949-bb42-4709-a0e8-2da4214e6135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.447046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3b949-bb42-4709-a0e8-2da4214e6135 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.448018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3b949-bb42-4709-a0e8-2da4214e6135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.451179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3b949-bb42-4709-a0e8-2da4214e6135 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:08.517995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 786/1580] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/projects => generated 405 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:08.519665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dba96-265b-4d18-b140-cf41004a4817 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.520065 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dba96-265b-4d18-b140-cf41004a4817 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.520582 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10dba96-265b-4d18-b140-cf41004a4817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.553129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a10dba96-265b-4d18-b140-cf41004a4817 None None] MFA Rules not processed for user `410cef275ee74b18afdbce7439e3ed41`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:08.554366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a10dba96-265b-4d18-b140-cf41004a4817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:08.604203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 795/1581] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:08.605562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3b949-bb42-4709-a0e8-2da4214e6135 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.607378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:08.634028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 787/1582] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:08.634889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dba96-265b-4d18-b140-cf41004a4817 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.697387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3b949-bb42-4709-a0e8-2da4214e6135 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.705124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07399c5e-affd-4e9f-9042-41ca37dfa99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.706096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07399c5e-affd-4e9f-9042-41ca37dfa99c admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.706661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07399c5e-affd-4e9f-9042-41ca37dfa99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.707003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07399c5e-affd-4e9f-9042-41ca37dfa99c admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.707818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07399c5e-affd-4e9f-9042-41ca37dfa99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.710188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07399c5e-affd-4e9f-9042-41ca37dfa99c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:08.724276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:08.734726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10dba96-265b-4d18-b140-cf41004a4817 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.741526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107e10f-42fe-4b5a-af42-5068002975d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.748009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107e10f-42fe-4b5a-af42-5068002975d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.748009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107e10f-42fe-4b5a-af42-5068002975d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.748009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107e10f-42fe-4b5a-af42-5068002975d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.748009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107e10f-42fe-4b5a-af42-5068002975d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.748009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107e10f-42fe-4b5a-af42-5068002975d5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:08.837807 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 796/1583] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:08.838662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07399c5e-affd-4e9f-9042-41ca37dfa99c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.851133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07399c5e-affd-4e9f-9042-41ca37dfa99c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.856144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2dc0f-7142-427a-a187-e319e68af03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.856816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2dc0f-7142-427a-a187-e319e68af03d admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.857101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2dc0f-7142-427a-a187-e319e68af03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.857369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2dc0f-7142-427a-a187-e319e68af03d admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.858007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2dc0f-7142-427a-a187-e319e68af03d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.859980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2dc0f-7142-427a-a187-e319e68af03d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:08.916680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 788/1584] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/users => generated 294 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:08.917832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107e10f-42fe-4b5a-af42-5068002975d5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.930186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107e10f-42fe-4b5a-af42-5068002975d5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.935237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d834c193-87c9-4526-8f15-b93026ce6cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.935843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834c193-87c9-4526-8f15-b93026ce6cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.936127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834c193-87c9-4526-8f15-b93026ce6cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.936467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834c193-87c9-4526-8f15-b93026ce6cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.937130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834c193-87c9-4526-8f15-b93026ce6cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.938793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d834c193-87c9-4526-8f15-b93026ce6cf1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:08.958857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 789/1585] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:08.958857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326bafb-128e-4590-b0d9-70489e06c150 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.958857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326bafb-128e-4590-b0d9-70489e06c150 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.958857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326bafb-128e-4590-b0d9-70489e06c150 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.959841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 790/1586] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 13:38:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:08.960769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326bafb-128e-4590-b0d9-70489e06c150 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:08.976097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326bafb-128e-4590-b0d9-70489e06c150 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:08.981020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161d167-d309-44c0-b715-f9c2a53a2afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:08.981643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d167-d309-44c0-b715-f9c2a53a2afb admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:08.981939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d167-d309-44c0-b715-f9c2a53a2afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:08.982202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d167-d309-44c0-b715-f9c2a53a2afb admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:08.982843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161d167-d309-44c0-b715-f9c2a53a2afb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:08.984916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161d167-d309-44c0-b715-f9c2a53a2afb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.029828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 797/1587] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:09.030916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2dc0f-7142-427a-a187-e319e68af03d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.046314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2dc0f-7142-427a-a187-e319e68af03d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.058349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85deed78-00be-481d-a59c-3e7cbb33f1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.059478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85deed78-00be-481d-a59c-3e7cbb33f1ce admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.060013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85deed78-00be-481d-a59c-3e7cbb33f1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.060589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85deed78-00be-481d-a59c-3e7cbb33f1ce admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.061825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85deed78-00be-481d-a59c-3e7cbb33f1ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.065851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85deed78-00be-481d-a59c-3e7cbb33f1ce admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.146271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 791/1588] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:08 2022] POST /identity/v3/users => generated 303 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:09.147209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161d167-d309-44c0-b715-f9c2a53a2afb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.158038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161d167-d309-44c0-b715-f9c2a53a2afb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.163515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89190581-c7d0-4ee5-a23e-b14e9997e0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.164142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190581-c7d0-4ee5-a23e-b14e9997e0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.164465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190581-c7d0-4ee5-a23e-b14e9997e0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.164739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89190581-c7d0-4ee5-a23e-b14e9997e0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.165391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89190581-c7d0-4ee5-a23e-b14e9997e0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.167449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89190581-c7d0-4ee5-a23e-b14e9997e0b4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.181549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 798/1589] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:09 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:09.181549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85deed78-00be-481d-a59c-3e7cbb33f1ce admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.189925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85deed78-00be-481d-a59c-3e7cbb33f1ce admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.195301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453fd6-d454-4368-83b4-b323f0b396a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.195930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453fd6-d454-4368-83b4-b323f0b396a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.196247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453fd6-d454-4368-83b4-b323f0b396a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.196533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453fd6-d454-4368-83b4-b323f0b396a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.197221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27453fd6-d454-4368-83b4-b323f0b396a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.198858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27453fd6-d454-4368-83b4-b323f0b396a7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.212669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 799/1590] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.212669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453fd6-d454-4368-83b4-b323f0b396a7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.223277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453fd6-d454-4368-83b4-b323f0b396a7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.228734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62b593-5063-4bed-b4d0-814197b212aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.229344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62b593-5063-4bed-b4d0-814197b212aa admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.229614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62b593-5063-4bed-b4d0-814197b212aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.229882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62b593-5063-4bed-b4d0-814197b212aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.230541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62b593-5063-4bed-b4d0-814197b212aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.232314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62b593-5063-4bed-b4d0-814197b212aa admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.244564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 800/1591] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.245854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62b593-5063-4bed-b4d0-814197b212aa admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.257428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62b593-5063-4bed-b4d0-814197b212aa admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.262233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c1a95-d66b-4bbd-b9fb-bf4c25e8d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.262887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1a95-d66b-4bbd-b9fb-bf4c25e8d7ae admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.263163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1a95-d66b-4bbd-b9fb-bf4c25e8d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.263427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c1a95-d66b-4bbd-b9fb-bf4c25e8d7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.264074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c1a95-d66b-4bbd-b9fb-bf4c25e8d7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.265725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c1a95-d66b-4bbd-b9fb-bf4c25e8d7ae admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.277700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 792/1592] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:09 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:09.279182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fe331-0da8-4556-bfc0-f595e2274f0d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.279562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fe331-0da8-4556-bfc0-f595e2274f0d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.279910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fe331-0da8-4556-bfc0-f595e2274f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.286962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 801/1593] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.287872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c1a95-d66b-4bbd-b9fb-bf4c25e8d7ae admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.300602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c1a95-d66b-4bbd-b9fb-bf4c25e8d7ae admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.331208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e81fe331-0da8-4556-bfc0-f595e2274f0d None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:09.333098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e81fe331-0da8-4556-bfc0-f595e2274f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:09.366264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af64ea-7fc4-479f-be14-4c2a6e90066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.367039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af64ea-7fc4-479f-be14-4c2a6e90066a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.367320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af64ea-7fc4-479f-be14-4c2a6e90066a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.367597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af64ea-7fc4-479f-be14-4c2a6e90066a admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.368289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af64ea-7fc4-479f-be14-4c2a6e90066a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.369908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af64ea-7fc4-479f-be14-4c2a6e90066a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.387803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 802/1594] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.387803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af64ea-7fc4-479f-be14-4c2a6e90066a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.399410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af64ea-7fc4-479f-be14-4c2a6e90066a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.404554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebc4d2-c6c2-47f5-a348-2a50be4b8844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.405343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc4d2-c6c2-47f5-a348-2a50be4b8844 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.405627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc4d2-c6c2-47f5-a348-2a50be4b8844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.405919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc4d2-c6c2-47f5-a348-2a50be4b8844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.406572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebc4d2-c6c2-47f5-a348-2a50be4b8844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.409039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebc4d2-c6c2-47f5-a348-2a50be4b8844 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.425140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 803/1595] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.425140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebc4d2-c6c2-47f5-a348-2a50be4b8844 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.428661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 793/1596] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 13:38:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:09.432291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebc4d2-c6c2-47f5-a348-2a50be4b8844 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.433813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fe331-0da8-4556-bfc0-f595e2274f0d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.437683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf880-566b-49df-ad0f-991c3fab2bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.438282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf880-566b-49df-ad0f-991c3fab2bae admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.438564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf880-566b-49df-ad0f-991c3fab2bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.438877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf880-566b-49df-ad0f-991c3fab2bae admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.439818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf880-566b-49df-ad0f-991c3fab2bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.441270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:09.441543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf880-566b-49df-ad0f-991c3fab2bae admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.447767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fe331-0da8-4556-bfc0-f595e2274f0d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.453790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 804/1597] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.457279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf880-566b-49df-ad0f-991c3fab2bae admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.458125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21747d7-dcda-43ad-b271-74a2cf22c034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.459268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21747d7-dcda-43ad-b271-74a2cf22c034 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.459987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21747d7-dcda-43ad-b271-74a2cf22c034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.460457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21747d7-dcda-43ad-b271-74a2cf22c034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.461244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21747d7-dcda-43ad-b271-74a2cf22c034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.463389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21747d7-dcda-43ad-b271-74a2cf22c034 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.466101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf880-566b-49df-ad0f-991c3fab2bae admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.471864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171368c-93a7-47d6-a069-adfac1790f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.472524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171368c-93a7-47d6-a069-adfac1790f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.472838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171368c-93a7-47d6-a069-adfac1790f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.473062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171368c-93a7-47d6-a069-adfac1790f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.473725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171368c-93a7-47d6-a069-adfac1790f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.475354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171368c-93a7-47d6-a069-adfac1790f95 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.480458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 794/1598] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.482570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21747d7-dcda-43ad-b271-74a2cf22c034 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.487721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 805/1599] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.491991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171368c-93a7-47d6-a069-adfac1790f95 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.500084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171368c-93a7-47d6-a069-adfac1790f95 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.502887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21747d7-dcda-43ad-b271-74a2cf22c034 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.506165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.506851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.507130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.507410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] PATH_INFO: `/v3/projects/4a3794670e8d4c9985547d557d6969e0/users/773dc87736034257b3d1ae2e8f02bf23/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.512578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8aca1-0297-4318-8943-80e941210a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.513834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8aca1-0297-4318-8943-80e941210a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.514654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8aca1-0297-4318-8943-80e941210a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.515127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8aca1-0297-4318-8943-80e941210a00 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.516016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8aca1-0297-4318-8943-80e941210a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.518784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8aca1-0297-4318-8943-80e941210a00 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.525265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3794670e8d4c9985547d557d6969e0, user_id=773dc87736034257b3d1ae2e8f02bf23, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.526332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:38:09.526332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Aug 24 13:38:09.526332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: Aug 24 13:38:09.528322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.641224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 806/1600] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:09 2022] PUT /identity/v3/projects/4a3794670e8d4c9985547d557d6969e0/users/773dc87736034257b3d1ae2e8f02bf23/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:09.642735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.672420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 795/1601] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:09 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:09.675320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8aca1-0297-4318-8943-80e941210a00 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.688390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:09.693900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8aca1-0297-4318-8943-80e941210a00 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.735735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb735f-0c8f-457d-bf82-d19b0f3a1baa admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.743461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c9200-e7aa-4aaa-ac5e-21dc551852a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.744481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c9200-e7aa-4aaa-ac5e-21dc551852a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.744934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c9200-e7aa-4aaa-ac5e-21dc551852a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.745381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c9200-e7aa-4aaa-ac5e-21dc551852a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.746284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c9200-e7aa-4aaa-ac5e-21dc551852a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.748412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c9200-e7aa-4aaa-ac5e-21dc551852a5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.765551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 807/1602] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.768868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c9200-e7aa-4aaa-ac5e-21dc551852a5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.792127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613ad5-180a-4067-b7b8-6fead9cfa21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.793157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613ad5-180a-4067-b7b8-6fead9cfa21f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.793606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613ad5-180a-4067-b7b8-6fead9cfa21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.794051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613ad5-180a-4067-b7b8-6fead9cfa21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.795189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4613ad5-180a-4067-b7b8-6fead9cfa21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.798148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4613ad5-180a-4067-b7b8-6fead9cfa21f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.818581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 796/1603] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.819479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613ad5-180a-4067-b7b8-6fead9cfa21f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.839501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613ad5-180a-4067-b7b8-6fead9cfa21f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.864043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edec4a3-8184-4e00-bb08-e64bbc7b44c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.864043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edec4a3-8184-4e00-bb08-e64bbc7b44c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.864043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edec4a3-8184-4e00-bb08-e64bbc7b44c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.864043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edec4a3-8184-4e00-bb08-e64bbc7b44c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.865862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edec4a3-8184-4e00-bb08-e64bbc7b44c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.865862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edec4a3-8184-4e00-bb08-e64bbc7b44c5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.866664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 797/1604] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:09.868586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edec4a3-8184-4e00-bb08-e64bbc7b44c5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:09.882954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edec4a3-8184-4e00-bb08-e64bbc7b44c5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.889507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46e3e2-6b23-40b4-8865-f49f3cec7f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.890501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46e3e2-6b23-40b4-8865-f49f3cec7f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.890915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46e3e2-6b23-40b4-8865-f49f3cec7f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.891327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46e3e2-6b23-40b4-8865-f49f3cec7f37 admin admin] PATH_INFO: `/v3/projects/77b5eebd2706440badffd858826d5c79/users/0108005591b14c5081d460f5c1a7526f/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.896065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c9200-e7aa-4aaa-ac5e-21dc551852a5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:09.903205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029994-bd2b-4ead-a7fa-843088bce17a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:09.903988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029994-bd2b-4ead-a7fa-843088bce17a service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:09.906335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029994-bd2b-4ead-a7fa-843088bce17a service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:09.906335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029994-bd2b-4ead-a7fa-843088bce17a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:09.914273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46e3e2-6b23-40b4-8865-f49f3cec7f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b5eebd2706440badffd858826d5c79, user_id=0108005591b14c5081d460f5c1a7526f, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.917620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46e3e2-6b23-40b4-8865-f49f3cec7f37 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.976003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c029994-bd2b-4ead-a7fa-843088bce17a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:09.977665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c029994-bd2b-4ead-a7fa-843088bce17a service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:09.989054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 808/1605] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:09.989919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029994-bd2b-4ead-a7fa-843088bce17a service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:10.001196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029994-bd2b-4ead-a7fa-843088bce17a service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:10.006519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99d2be-323d-490f-b16c-8d4bb79ceece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:10.007222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99d2be-323d-490f-b16c-8d4bb79ceece admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.007503 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99d2be-323d-490f-b16c-8d4bb79ceece admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.007786 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99d2be-323d-490f-b16c-8d4bb79ceece admin admin] PATH_INFO: `/v3/projects/5b1447b864ba4b2faae763b20bda8003/users/499975dba40446f9b28cc91a5c3f8ceb/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.025950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99d2be-323d-490f-b16c-8d4bb79ceece admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1447b864ba4b2faae763b20bda8003, user_id=499975dba40446f9b28cc91a5c3f8ceb, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:10.028833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99d2be-323d-490f-b16c-8d4bb79ceece admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:10.062601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 798/1606] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:09 2022] PUT /identity/v3/projects/77b5eebd2706440badffd858826d5c79/users/0108005591b14c5081d460f5c1a7526f/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:10.065200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0abb-ec2c-41c3-8c65-80fe6955e8de None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.065768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0abb-ec2c-41c3-8c65-80fe6955e8de None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.066294 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf0abb-ec2c-41c3-8c65-80fe6955e8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.089710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3aaf0abb-ec2c-41c3-8c65-80fe6955e8de None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:10.091062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3aaf0abb-ec2c-41c3-8c65-80fe6955e8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:10.150841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 809/1607] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:09 2022] PUT /identity/v3/projects/5b1447b864ba4b2faae763b20bda8003/users/499975dba40446f9b28cc91a5c3f8ceb/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:10.154733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99d2be-323d-490f-b16c-8d4bb79ceece admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:10.169686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 799/1608] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:10.176812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf0abb-ec2c-41c3-8c65-80fe6955e8de None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:10.192108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf0abb-ec2c-41c3-8c65-80fe6955e8de None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:10.233855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:10.238547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:10.274916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99d2be-323d-490f-b16c-8d4bb79ceece admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:10.280324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde7868-36e4-46a0-b270-b44bd7f7be6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:10.285954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff53460-7db6-4dc1-ad9b-405b0c80da9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:10.286138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7868-36e4-46a0-b270-b44bd7f7be6f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.286138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7868-36e4-46a0-b270-b44bd7f7be6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.286138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7868-36e4-46a0-b270-b44bd7f7be6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.286502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff53460-7db6-4dc1-ad9b-405b0c80da9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.286592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde7868-36e4-46a0-b270-b44bd7f7be6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:10.286679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff53460-7db6-4dc1-ad9b-405b0c80da9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.286679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff53460-7db6-4dc1-ad9b-405b0c80da9b admin admin] PATH_INFO: `/v3/projects/6be8ac36241e4131b3846cd43f01a3b4/users/f924d125ffa245d1a34ccdc160c7b86b/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.286860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde7868-36e4-46a0-b270-b44bd7f7be6f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:10.297917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 810/1609] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:10.303099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde7868-36e4-46a0-b270-b44bd7f7be6f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:10.311054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff53460-7db6-4dc1-ad9b-405b0c80da9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be8ac36241e4131b3846cd43f01a3b4, user_id=f924d125ffa245d1a34ccdc160c7b86b, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:10.317641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff53460-7db6-4dc1-ad9b-405b0c80da9b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:10.317879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde7868-36e4-46a0-b270-b44bd7f7be6f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:10.323637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69ad15-e234-4dfb-ae21-a1d687e83175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:10.324655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ad15-e234-4dfb-ae21-a1d687e83175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.325059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ad15-e234-4dfb-ae21-a1d687e83175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.325515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69ad15-e234-4dfb-ae21-a1d687e83175 admin admin] PATH_INFO: `/v3/projects/c7eed89ca88b43ab98e653fe4806ef26/users/6553f580ee7a492383982fc71a183b31/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.343002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69ad15-e234-4dfb-ae21-a1d687e83175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7eed89ca88b43ab98e653fe4806ef26, user_id=6553f580ee7a492383982fc71a183b31, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:10.347039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69ad15-e234-4dfb-ae21-a1d687e83175 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:10.449954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 800/1610] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:10 2022] PUT /identity/v3/projects/6be8ac36241e4131b3846cd43f01a3b4/users/f924d125ffa245d1a34ccdc160c7b86b/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:10.460714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff53460-7db6-4dc1-ad9b-405b0c80da9b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:10.538444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 811/1611] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:10 2022] PUT /identity/v3/projects/c7eed89ca88b43ab98e653fe4806ef26/users/6553f580ee7a492383982fc71a183b31/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:10.539707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d112d0-4cfc-4652-9dc3-412d1f9ec52e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.539996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d112d0-4cfc-4652-9dc3-412d1f9ec52e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.540321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d112d0-4cfc-4652-9dc3-412d1f9ec52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.542530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff53460-7db6-4dc1-ad9b-405b0c80da9b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:10.572969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a1d112d0-4cfc-4652-9dc3-412d1f9ec52e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:10.574223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a1d112d0-4cfc-4652-9dc3-412d1f9ec52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:10.622609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:10.623650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.624146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.624572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] PATH_INFO: `/v3/projects/019b2e8bdb934d94a25fe4b355e73c68/users/ea9d4eafe59c4c8180eece8ec9283e40/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.644433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=019b2e8bdb934d94a25fe4b355e73c68, user_id=ea9d4eafe59c4c8180eece8ec9283e40, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:10.649510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:10.743761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 812/1612] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:10.746305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2d2f0-9421-4a58-ae81-a7aa0e9c7159 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.746893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2d2f0-9421-4a58-ae81-a7aa0e9c7159 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.747577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2d2f0-9421-4a58-ae81-a7aa0e9c7159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.784141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-81d2d2f0-9421-4a58-ae81-a7aa0e9c7159 None None] MFA Rules not processed for user `499975dba40446f9b28cc91a5c3f8ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:10.785538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-81d2d2f0-9421-4a58-ae81-a7aa0e9c7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:10.787850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 801/1613] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:10 2022] PUT /identity/v3/projects/019b2e8bdb934d94a25fe4b355e73c68/users/ea9d4eafe59c4c8180eece8ec9283e40/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:10.792488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:10.792488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:10.858460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:10.863869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25e928-3869-4c01-a7f1-e8a565cc5ee2 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:10.868108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06d518-1586-4ea5-a005-c85a2ccfd61c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:10.869037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06d518-1586-4ea5-a005-c85a2ccfd61c service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.869589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06d518-1586-4ea5-a005-c85a2ccfd61c service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.869949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06d518-1586-4ea5-a005-c85a2ccfd61c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.871912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 813/1614] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:10.872573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2d2f0-9421-4a58-ae81-a7aa0e9c7159 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:10.917938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:10.943283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06d518-1586-4ea5-a005-c85a2ccfd61c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:10.944870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06d518-1586-4ea5-a005-c85a2ccfd61c service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:10.945117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2d2f0-9421-4a58-ae81-a7aa0e9c7159 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:10.950142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33272e9-7c02-4c16-8c2f-1187acbb5ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:10.950816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33272e9-7c02-4c16-8c2f-1187acbb5ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.951106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33272e9-7c02-4c16-8c2f-1187acbb5ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.951384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33272e9-7c02-4c16-8c2f-1187acbb5ccf admin admin] PATH_INFO: `/v3/projects/298640606eb344698dd46b8b60e2a515/users/49ec9aca9e8d47578c11dcd742bf0ae5/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.957392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 802/1615] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:10.959780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ebaf-af1b-48de-8ee0-cd24e156fc25 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:10.960295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ebaf-af1b-48de-8ee0-cd24e156fc25 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:10.960973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5ebaf-af1b-48de-8ee0-cd24e156fc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:10.971058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33272e9-7c02-4c16-8c2f-1187acbb5ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=298640606eb344698dd46b8b60e2a515, user_id=49ec9aca9e8d47578c11dcd742bf0ae5, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:10.974042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33272e9-7c02-4c16-8c2f-1187acbb5ccf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:11.007767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-07e5ebaf-af1b-48de-8ee0-cd24e156fc25 None None] MFA Rules not processed for user `f924d125ffa245d1a34ccdc160c7b86b`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:11.009216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-07e5ebaf-af1b-48de-8ee0-cd24e156fc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:11.074243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 814/1616] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:10 2022] PUT /identity/v3/projects/298640606eb344698dd46b8b60e2a515/users/49ec9aca9e8d47578c11dcd742bf0ae5/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:11.077091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d22ef-b27b-40fd-8e04-ff37d9383dcc None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.077503 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d22ef-b27b-40fd-8e04-ff37d9383dcc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.078036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d22ef-b27b-40fd-8e04-ff37d9383dcc None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.081864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 815/1617] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 13:38:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:11.084113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3482e-1013-46ba-b8ea-ed35214d83a8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.084533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3482e-1013-46ba-b8ea-ed35214d83a8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.085041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3482e-1013-46ba-b8ea-ed35214d83a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.094974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 803/1618] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:11.097116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5ebaf-af1b-48de-8ee0-cd24e156fc25 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:11.117464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5ebaf-af1b-48de-8ee0-cd24e156fc25 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:11.141428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9fa3482e-1013-46ba-b8ea-ed35214d83a8 None None] MFA Rules not processed for user `6553f580ee7a492383982fc71a183b31`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:11.144989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9fa3482e-1013-46ba-b8ea-ed35214d83a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:11.223867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db4c15-84b7-4889-b8d0-610f166c364d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:11.224684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db4c15-84b7-4889-b8d0-610f166c364d service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.224964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db4c15-84b7-4889-b8d0-610f166c364d service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.225342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db4c15-84b7-4889-b8d0-610f166c364d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.253528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 816/1619] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:11 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:11.256130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cbfd9-0b14-4e51-89d5-3268839b73f3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.256918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cbfd9-0b14-4e51-89d5-3268839b73f3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.259102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cbfd9-0b14-4e51-89d5-3268839b73f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.283443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-779cbfd9-0b14-4e51-89d5-3268839b73f3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:11.285203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-779cbfd9-0b14-4e51-89d5-3268839b73f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:11.286762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:11.321730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db4c15-84b7-4889-b8d0-610f166c364d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:11.323611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db4c15-84b7-4889-b8d0-610f166c364d service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:11.337753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 804/1620] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:11.339425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db4c15-84b7-4889-b8d0-610f166c364d service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:11.360043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db4c15-84b7-4889-b8d0-610f166c364d service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:11.371845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcff260-9580-416a-8c9e-b2075f5f78d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:11.373231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff260-9580-416a-8c9e-b2075f5f78d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.373788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff260-9580-416a-8c9e-b2075f5f78d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.374305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff260-9580-416a-8c9e-b2075f5f78d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.375548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcff260-9580-416a-8c9e-b2075f5f78d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:11.379373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 817/1621] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:11.380733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cbfd9-0b14-4e51-89d5-3268839b73f3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:11.381939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcff260-9580-416a-8c9e-b2075f5f78d0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:11.407042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cbfd9-0b14-4e51-89d5-3268839b73f3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:11.416891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f7c6a-77ce-4998-b6c2-3d636057a282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:11.418063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7c6a-77ce-4998-b6c2-3d636057a282 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.419771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7c6a-77ce-4998-b6c2-3d636057a282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.419771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7c6a-77ce-4998-b6c2-3d636057a282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.419771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f7c6a-77ce-4998-b6c2-3d636057a282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:11.421700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f7c6a-77ce-4998-b6c2-3d636057a282 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:11.440676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 818/1622] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:11.440676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaff8c-8a75-4f97-9608-c19303aeb096 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.440676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaff8c-8a75-4f97-9608-c19303aeb096 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.441279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaff8c-8a75-4f97-9608-c19303aeb096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.467878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fbcaff8c-8a75-4f97-9608-c19303aeb096 None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:11.469591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fbcaff8c-8a75-4f97-9608-c19303aeb096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:11.519062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:11.560272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 805/1623] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:11 2022] POST /identity/v3/projects => generated 387 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:11.567031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff6a8e-5890-4e30-acbe-73927d06191b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.567361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff6a8e-5890-4e30-acbe-73927d06191b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.567618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff6a8e-5890-4e30-acbe-73927d06191b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.569057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 806/1624] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 13:38:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:11.569815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff6a8e-5890-4e30-acbe-73927d06191b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:11.612393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 819/1625] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:38:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:11.618793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcaff8c-8a75-4f97-9608-c19303aeb096 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:11.634904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcaff8c-8a75-4f97-9608-c19303aeb096 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:11.709417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff6a8e-5890-4e30-acbe-73927d06191b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:11.709591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa6075-2393-4ee4-a0df-d4d4bf8d112b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:11.710359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa6075-2393-4ee4-a0df-d4d4bf8d112b service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.710704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa6075-2393-4ee4-a0df-d4d4bf8d112b service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.710934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa6075-2393-4ee4-a0df-d4d4bf8d112b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.713411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464af89-fe26-4178-8b51-04545d181484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:11.714786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464af89-fe26-4178-8b51-04545d181484 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.715506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464af89-fe26-4178-8b51-04545d181484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.716223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464af89-fe26-4178-8b51-04545d181484 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.717352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464af89-fe26-4178-8b51-04545d181484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:11.721315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464af89-fe26-4178-8b51-04545d181484 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:11.725317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa6075-2393-4ee4-a0df-d4d4bf8d112b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:11.729852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa6075-2393-4ee4-a0df-d4d4bf8d112b service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:11.756243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 820/1626] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:11.758232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa6075-2393-4ee4-a0df-d4d4bf8d112b service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:11.773581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa6075-2393-4ee4-a0df-d4d4bf8d112b service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:11.781037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f4064-e01e-4a70-8363-12a2fe05e620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:11.781891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f4064-e01e-4a70-8363-12a2fe05e620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.782233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f4064-e01e-4a70-8363-12a2fe05e620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.782522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f4064-e01e-4a70-8363-12a2fe05e620 admin admin] PATH_INFO: `/v3/projects/298640606eb344698dd46b8b60e2a515/users/49ec9aca9e8d47578c11dcd742bf0ae5/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.785815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f4064-e01e-4a70-8363-12a2fe05e620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=298640606eb344698dd46b8b60e2a515, user_id=49ec9aca9e8d47578c11dcd742bf0ae5, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:11.788801 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f4064-e01e-4a70-8363-12a2fe05e620 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:11.802224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 821/1627] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:11 2022] PUT /identity/v3/projects/298640606eb344698dd46b8b60e2a515/users/49ec9aca9e8d47578c11dcd742bf0ae5/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:11.811357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f4064-e01e-4a70-8363-12a2fe05e620 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:11.910048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 807/1628] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:11 2022] POST /identity/v3/projects => generated 405 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:11.914455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193440-812f-4b3c-8085-8f2c617fd8f7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:11.914914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193440-812f-4b3c-8085-8f2c617fd8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:11.915351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193440-812f-4b3c-8085-8f2c617fd8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:11.959340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2b193440-812f-4b3c-8085-8f2c617fd8f7 None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:11.960646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2b193440-812f-4b3c-8085-8f2c617fd8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:11.988729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f4064-e01e-4a70-8363-12a2fe05e620 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.109194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1226e-4528-47d4-9540-d8540a5315e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.111710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1226e-4528-47d4-9540-d8540a5315e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.112112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1226e-4528-47d4-9540-d8540a5315e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.112784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1226e-4528-47d4-9540-d8540a5315e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.114986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1226e-4528-47d4-9540-d8540a5315e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.119678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1226e-4528-47d4-9540-d8540a5315e5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:12.146541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:12.155663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 808/1629] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 13:38:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:12.157669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b193440-812f-4b3c-8085-8f2c617fd8f7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:12.233357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:12.268674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b193440-812f-4b3c-8085-8f2c617fd8f7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.277665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac453de-6cdf-4fab-82ff-78546ad4e15d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.278741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac453de-6cdf-4fab-82ff-78546ad4e15d service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.279366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac453de-6cdf-4fab-82ff-78546ad4e15d service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.279745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac453de-6cdf-4fab-82ff-78546ad4e15d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.287312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 822/1630] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:11 2022] POST /identity/v3/users => generated 300 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:12.287312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9168fac-ed00-49ae-b1b3-8f0432612346 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.287312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9168fac-ed00-49ae-b1b3-8f0432612346 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.287312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9168fac-ed00-49ae-b1b3-8f0432612346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.304821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac453de-6cdf-4fab-82ff-78546ad4e15d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.306483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac453de-6cdf-4fab-82ff-78546ad4e15d service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:12.325458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 809/1631] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:12.326939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac453de-6cdf-4fab-82ff-78546ad4e15d service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:12.339449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac453de-6cdf-4fab-82ff-78546ad4e15d service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.345375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a9168fac-ed00-49ae-b1b3-8f0432612346 None None] MFA Rules not processed for user `49ec9aca9e8d47578c11dcd742bf0ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:12.347115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a9168fac-ed00-49ae-b1b3-8f0432612346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:12.354565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafcd0d-2077-4eee-91fd-0ce12fdedb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.358813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafcd0d-2077-4eee-91fd-0ce12fdedb2e admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.358813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafcd0d-2077-4eee-91fd-0ce12fdedb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.358813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafcd0d-2077-4eee-91fd-0ce12fdedb2e admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.358813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafcd0d-2077-4eee-91fd-0ce12fdedb2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.363028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafcd0d-2077-4eee-91fd-0ce12fdedb2e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:12.454291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 823/1632] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:12.455968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9168fac-ed00-49ae-b1b3-8f0432612346 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:12.494175 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9168fac-ed00-49ae-b1b3-8f0432612346 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.499036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 810/1633] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:12 2022] POST /identity/v3/users => generated 309 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:12.502265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafcd0d-2077-4eee-91fd-0ce12fdedb2e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:12.506928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408e588-79f7-4cb3-bf53-5e9b1327f5eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.507816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408e588-79f7-4cb3-bf53-5e9b1327f5eb service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.508111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408e588-79f7-4cb3-bf53-5e9b1327f5eb service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.508481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408e588-79f7-4cb3-bf53-5e9b1327f5eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.537074 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafcd0d-2077-4eee-91fd-0ce12fdedb2e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.548482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653c0f3-a2ec-4e21-83bd-bd6e368b83af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.549984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653c0f3-a2ec-4e21-83bd-bd6e368b83af admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.551020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653c0f3-a2ec-4e21-83bd-bd6e368b83af admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.551641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653c0f3-a2ec-4e21-83bd-bd6e368b83af admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.553162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653c0f3-a2ec-4e21-83bd-bd6e368b83af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.557351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653c0f3-a2ec-4e21-83bd-bd6e368b83af admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:12.584792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 811/1634] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:12.587080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653c0f3-a2ec-4e21-83bd-bd6e368b83af admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:12.618431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653c0f3-a2ec-4e21-83bd-bd6e368b83af admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.621894 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408e588-79f7-4cb3-bf53-5e9b1327f5eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.624156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08f8fe-f2d2-4e62-bb14-2d0530ee09e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.625168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08f8fe-f2d2-4e62-bb14-2d0530ee09e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.625666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08f8fe-f2d2-4e62-bb14-2d0530ee09e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.626137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08f8fe-f2d2-4e62-bb14-2d0530ee09e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.628641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408e588-79f7-4cb3-bf53-5e9b1327f5eb service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:12.653590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08f8fe-f2d2-4e62-bb14-2d0530ee09e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.659293 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08f8fe-f2d2-4e62-bb14-2d0530ee09e6 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:12.662980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 824/1635] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:12.665746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408e588-79f7-4cb3-bf53-5e9b1327f5eb service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:12.693505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 812/1636] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:12.698410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08f8fe-f2d2-4e62-bb14-2d0530ee09e6 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:12.723121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408e588-79f7-4cb3-bf53-5e9b1327f5eb service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.726306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08f8fe-f2d2-4e62-bb14-2d0530ee09e6 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.741298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046505db-a3f7-45c7-aa66-9693b7dad6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.743218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046505db-a3f7-45c7-aa66-9693b7dad6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.744004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046505db-a3f7-45c7-aa66-9693b7dad6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.745920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046505db-a3f7-45c7-aa66-9693b7dad6cc admin admin] PATH_INFO: `/v3/projects/4df893ed17604aa78f8367737e3b5cf5/users/3fa98e3909134510a837775aea25d406/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.747507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1be7aa-2ed1-4f24-949e-a7728c17c619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.748825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be7aa-2ed1-4f24-949e-a7728c17c619 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.749313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be7aa-2ed1-4f24-949e-a7728c17c619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.751875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be7aa-2ed1-4f24-949e-a7728c17c619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.755077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1be7aa-2ed1-4f24-949e-a7728c17c619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.759356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1be7aa-2ed1-4f24-949e-a7728c17c619 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:12.788748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 825/1637] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:12.796993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1be7aa-2ed1-4f24-949e-a7728c17c619 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:12.813628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046505db-a3f7-45c7-aa66-9693b7dad6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df893ed17604aa78f8367737e3b5cf5, user_id=3fa98e3909134510a837775aea25d406, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.823156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1be7aa-2ed1-4f24-949e-a7728c17c619 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:12.832828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb99c6c-2f69-4e3c-9d68-3e3de53335ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:12.834275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb99c6c-2f69-4e3c-9d68-3e3de53335ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:12.834915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb99c6c-2f69-4e3c-9d68-3e3de53335ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:12.835514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb99c6c-2f69-4e3c-9d68-3e3de53335ce admin admin] PATH_INFO: `/v3/projects/4f94d3a505fb4a00b8e705b0e200b3f8/users/b327742cfd674637a53a81a4c6e198e4/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:12.837236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046505db-a3f7-45c7-aa66-9693b7dad6cc admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:12.905624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb99c6c-2f69-4e3c-9d68-3e3de53335ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f94d3a505fb4a00b8e705b0e200b3f8, user_id=b327742cfd674637a53a81a4c6e198e4, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:12.911757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb99c6c-2f69-4e3c-9d68-3e3de53335ce admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:13.077120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 813/1638] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:12 2022] PUT /identity/v3/projects/4df893ed17604aa78f8367737e3b5cf5/users/3fa98e3909134510a837775aea25d406/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:13.084513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046505db-a3f7-45c7-aa66-9693b7dad6cc admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:13.120352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 826/1639] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:12 2022] PUT /identity/v3/projects/4f94d3a505fb4a00b8e705b0e200b3f8/users/b327742cfd674637a53a81a4c6e198e4/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:13.122764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb99c6c-2f69-4e3c-9d68-3e3de53335ce admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:13.255952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046505db-a3f7-45c7-aa66-9693b7dad6cc admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:13.281419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:13.327749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb99c6c-2f69-4e3c-9d68-3e3de53335ce admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:13.340540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73194277-eb06-4526-858d-e13930b2a97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:13.342436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73194277-eb06-4526-858d-e13930b2a97d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:13.343294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73194277-eb06-4526-858d-e13930b2a97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:13.344843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73194277-eb06-4526-858d-e13930b2a97d admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:13.348387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73194277-eb06-4526-858d-e13930b2a97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:13.354667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73194277-eb06-4526-858d-e13930b2a97d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:13.380630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 827/1640] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:13.382837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8aa2e7-55b8-4e7a-a518-a1fb6beb5e1f None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:13.383228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8aa2e7-55b8-4e7a-a518-a1fb6beb5e1f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:13.384491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8aa2e7-55b8-4e7a-a518-a1fb6beb5e1f None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:13.387693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 828/1641] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 13:38:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:13.395871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8aa2e7-55b8-4e7a-a518-a1fb6beb5e1f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:13.401978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64e7b8-1569-4ca9-a4ef-ce53caa2019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:13.403872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e7b8-1569-4ca9-a4ef-ce53caa2019d admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:13.407079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e7b8-1569-4ca9-a4ef-ce53caa2019d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:13.409022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e7b8-1569-4ca9-a4ef-ce53caa2019d admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:13.413095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64e7b8-1569-4ca9-a4ef-ce53caa2019d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:13.421750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64e7b8-1569-4ca9-a4ef-ce53caa2019d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:13.452714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 814/1642] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:38:13.452714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2c7ea-d0ef-4ede-ae27-062edaf92c0e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:13.452714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2c7ea-d0ef-4ede-ae27-062edaf92c0e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:13.452714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2c7ea-d0ef-4ede-ae27-062edaf92c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:13.486318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8aa2e7-55b8-4e7a-a518-a1fb6beb5e1f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:13.500914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab49f16-7f9d-48dc-a617-0d42e53f570e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:13.503904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49f16-7f9d-48dc-a617-0d42e53f570e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:13.504398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49f16-7f9d-48dc-a617-0d42e53f570e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:13.504876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab49f16-7f9d-48dc-a617-0d42e53f570e admin admin] PATH_INFO: `/v3/projects/4f94d3a505fb4a00b8e705b0e200b3f8/users/b327742cfd674637a53a81a4c6e198e4/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:13.507952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab49f16-7f9d-48dc-a617-0d42e53f570e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f94d3a505fb4a00b8e705b0e200b3f8, user_id=b327742cfd674637a53a81a4c6e198e4, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:13.513667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab49f16-7f9d-48dc-a617-0d42e53f570e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:13.531575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 829/1643] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:13 2022] PUT /identity/v3/projects/4f94d3a505fb4a00b8e705b0e200b3f8/users/b327742cfd674637a53a81a4c6e198e4/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:13.533319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab49f16-7f9d-48dc-a617-0d42e53f570e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:13.547339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-72a2c7ea-d0ef-4ede-ae27-062edaf92c0e None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:13.552660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-72a2c7ea-d0ef-4ede-ae27-062edaf92c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:13.584602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:13.745036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab49f16-7f9d-48dc-a617-0d42e53f570e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:13.762990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e38dd-09a3-472b-9e37-480ca40dd1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:13.764232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 815/1644] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:13.764670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e38dd-09a3-472b-9e37-480ca40dd1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:13.765284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e38dd-09a3-472b-9e37-480ca40dd1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:13.765910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e38dd-09a3-472b-9e37-480ca40dd1bb admin admin] PATH_INFO: `/v3/projects/4df893ed17604aa78f8367737e3b5cf5/users/3fa98e3909134510a837775aea25d406/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:13.769799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d9429-0de5-4061-b44a-f992847eaabc None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:13.778089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d9429-0de5-4061-b44a-f992847eaabc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:13.778525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e38dd-09a3-472b-9e37-480ca40dd1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df893ed17604aa78f8367737e3b5cf5, user_id=3fa98e3909134510a837775aea25d406, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:13.778711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d9429-0de5-4061-b44a-f992847eaabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:13.778884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e38dd-09a3-472b-9e37-480ca40dd1bb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:13.808614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 830/1645] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:13 2022] PUT /identity/v3/projects/4df893ed17604aa78f8367737e3b5cf5/users/3fa98e3909134510a837775aea25d406/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:13.821180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe9d2d-e9eb-4674-9457-752c15630960 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:13.821610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe9d2d-e9eb-4674-9457-752c15630960 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:13.822030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe9d2d-e9eb-4674-9457-752c15630960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:13.861254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b86d9429-0de5-4061-b44a-f992847eaabc None None] MFA Rules not processed for user `b327742cfd674637a53a81a4c6e198e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:13.863716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b86d9429-0de5-4061-b44a-f992847eaabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:13.911371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d2fe9d2d-e9eb-4674-9457-752c15630960 None None] MFA Rules not processed for user `3fa98e3909134510a837775aea25d406`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:13.913910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d2fe9d2d-e9eb-4674-9457-752c15630960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:14.115298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 816/1646] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:14.117573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d9429-0de5-4061-b44a-f992847eaabc None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:14.216769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 831/1647] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:14.285326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d9429-0de5-4061-b44a-f992847eaabc None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:14.294408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534082-64df-4b35-aae9-d9fb8147054e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:14.296971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534082-64df-4b35-aae9-d9fb8147054e service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:14.297652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534082-64df-4b35-aae9-d9fb8147054e service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:14.298108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86534082-64df-4b35-aae9-d9fb8147054e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:14.317877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe9d2d-e9eb-4674-9457-752c15630960 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:14.335580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe9d2d-e9eb-4674-9457-752c15630960 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:14.342173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447a476-adfc-4f4a-8ebf-0f72224a5fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:14.343602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447a476-adfc-4f4a-8ebf-0f72224a5fdb service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:14.344552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447a476-adfc-4f4a-8ebf-0f72224a5fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:14.346032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447a476-adfc-4f4a-8ebf-0f72224a5fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:14.368075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447a476-adfc-4f4a-8ebf-0f72224a5fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:14.371189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447a476-adfc-4f4a-8ebf-0f72224a5fdb service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:14.393130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 832/1648] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:14 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:38:14.444489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:14.519964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86534082-64df-4b35-aae9-d9fb8147054e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:14.521588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86534082-64df-4b35-aae9-d9fb8147054e service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:14.558084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 817/1649] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:14.642709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447a476-adfc-4f4a-8ebf-0f72224a5fdb service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:14.667565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447a476-adfc-4f4a-8ebf-0f72224a5fdb service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:14.701661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e822885-91c1-4831-b2b9-d74a2ff017b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:14.704255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e822885-91c1-4831-b2b9-d74a2ff017b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:14.705864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e822885-91c1-4831-b2b9-d74a2ff017b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:14.707758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e822885-91c1-4831-b2b9-d74a2ff017b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:14.769229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e822885-91c1-4831-b2b9-d74a2ff017b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:14.779308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e822885-91c1-4831-b2b9-d74a2ff017b6 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:14.821795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 833/1650] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:15.519529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534082-64df-4b35-aae9-d9fb8147054e service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:15.813796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86534082-64df-4b35-aae9-d9fb8147054e service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:15.833373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:15.833840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:15.834298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:15.835604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] PATH_INFO: `/v3/users/499975dba40446f9b28cc91a5c3f8ceb` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:15.837417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=499975dba40446f9b28cc91a5c3f8ceb)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:15.844165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:16.222617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e822885-91c1-4831-b2b9-d74a2ff017b6 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:16.298963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e822885-91c1-4831-b2b9-d74a2ff017b6 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:16.323586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:16.328556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:16.329333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499975dba40446f9b28cc91a5c3f8ceb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.331271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:16.333514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499975dba40446f9b28cc91a5c3f8ceb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.334066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] PATH_INFO: `/v3/users/6553f580ee7a492383982fc71a183b31` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:16.341949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6553f580ee7a492383982fc71a183b31)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:16.350745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: WARNING py.warnings [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:38:16.350745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: warnings.warn(msg) Aug 24 13:38:16.350745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: Aug 24 13:38:16.358005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:16.379035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499975dba40446f9b28cc91a5c3f8ceb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.386686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499975dba40446f9b28cc91a5c3f8ceb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.439824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:16.536123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499975dba40446f9b28cc91a5c3f8ceb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.602506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 818/1651] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:15 2022] DELETE /identity/v3/users/499975dba40446f9b28cc91a5c3f8ceb => generated 0 bytes in 1086 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:16.606391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:16.610729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:16.769870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6553f580ee7a492383982fc71a183b31'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.770783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6553f580ee7a492383982fc71a183b31'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.803699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6553f580ee7a492383982fc71a183b31'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.886307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:16.938264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7588-e5c5-4a51-96f4-88db0b8a1d3c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:16.952577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6553f580ee7a492383982fc71a183b31'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.953317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-19972e95-dfe5-48ec-aa62-ccb58592305e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6553f580ee7a492383982fc71a183b31'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:16.967740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:16.969247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:16.970006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:16.971606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] PATH_INFO: `/v3/users/410cef275ee74b18afdbce7439e3ed41` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:16.975912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=410cef275ee74b18afdbce7439e3ed41)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:16.979854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:17.035231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 834/1652] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:16 2022] DELETE /identity/v3/users/6553f580ee7a492383982fc71a183b31 => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:17.036848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727b8f0-a49e-4f62-a1e7-74d0fd8f0e27 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:17.037277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727b8f0-a49e-4f62-a1e7-74d0fd8f0e27 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:17.037964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727b8f0-a49e-4f62-a1e7-74d0fd8f0e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:17.080503 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b727b8f0-a49e-4f62-a1e7-74d0fd8f0e27 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:17.083035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b727b8f0-a49e-4f62-a1e7-74d0fd8f0e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:17.218167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 835/1653] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:17.219824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182097ae-7079-4faa-a41b-1ff9b082f36b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:17.221440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182097ae-7079-4faa-a41b-1ff9b082f36b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:17.222301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182097ae-7079-4faa-a41b-1ff9b082f36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:17.256675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-182097ae-7079-4faa-a41b-1ff9b082f36b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:17.256675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-182097ae-7079-4faa-a41b-1ff9b082f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:17.280103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 836/1654] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:17.281122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182097ae-7079-4faa-a41b-1ff9b082f36b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:17.284408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-182097ae-7079-4faa-a41b-1ff9b082f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:17.387797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182097ae-7079-4faa-a41b-1ff9b082f36b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:17.395264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '410cef275ee74b18afdbce7439e3ed41'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:17.396425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '410cef275ee74b18afdbce7439e3ed41'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:17.398470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:17.399767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:17.400519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:17.401009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:17.406714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:17.409147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '410cef275ee74b18afdbce7439e3ed41'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:17.410880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '410cef275ee74b18afdbce7439e3ed41'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:17.455258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:17.456794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:17.482578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 837/1655] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:17.483251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:17.486466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:17.522907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69123c-5452-46c0-b983-a064ff66ed9d service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:17.553745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9ab05e93-f6b7-43d1-9876-267dab2d1b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '410cef275ee74b18afdbce7439e3ed41'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:17.564050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:17.564875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:17.565166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:17.565439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:17.574678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:17.603996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 819/1656] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:16 2022] DELETE /identity/v3/users/410cef275ee74b18afdbce7439e3ed41 => generated 0 bytes in 1001 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:17.624262 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:17.628261 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038940d4-d86c-4337-b8ae-ace1e226d4d3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:17.628462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:17.629104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038940d4-d86c-4337-b8ae-ace1e226d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:17.636984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038940d4-d86c-4337-b8ae-ace1e226d4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:17.670732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-038940d4-d86c-4337-b8ae-ace1e226d4d3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:17.672569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-038940d4-d86c-4337-b8ae-ace1e226d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:17.703650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 838/1657] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:17.870766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 820/1658] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:17.984716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:17.986338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:18.004478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f785035-98f3-4498-aa06-9042b94a995c service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:18.141667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dc97a-7016-4778-8530-edf9028dbfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:18.143410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dc97a-7016-4778-8530-edf9028dbfbc service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:18.143603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dc97a-7016-4778-8530-edf9028dbfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:18.144003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dc97a-7016-4778-8530-edf9028dbfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:18.157853 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dc97a-7016-4778-8530-edf9028dbfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:18.161353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dc97a-7016-4778-8530-edf9028dbfbc service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:18.210789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 839/1659] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:19.380473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:19.706304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038940d4-d86c-4337-b8ae-ace1e226d4d3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:19.711474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-038940d4-d86c-4337-b8ae-ace1e226d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:19.730337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038940d4-d86c-4337-b8ae-ace1e226d4d3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:19.741456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:19.741456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:19.741456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:19.741456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] PATH_INFO: `/v3/projects/c7eed89ca88b43ab98e653fe4806ef26` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:19.742541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7eed89ca88b43ab98e653fe4806ef26)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:19.744920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: WARNING py.warnings [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 13:38:19.744920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: warnings.warn(msg) Aug 24 13:38:19.744920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: Aug 24 13:38:19.746859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:19.914380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7eed89ca88b43ab98e653fe4806ef26', 'request_id': 'req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:19.926750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7eed89ca88b43ab98e653fe4806ef26', 'request_id': 'req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:19.956717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7eed89ca88b43ab98e653fe4806ef26', 'request_id': 'req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:19.970950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:20.065922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] The token cache is being invalidate because project c7eed89ca88b43ab98e653fe4806ef26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:20.066170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.068947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.075061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 821/1660] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:19 2022] DELETE /identity/v3/projects/c7eed89ca88b43ab98e653fe4806ef26 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:20.076292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dc97a-7016-4778-8530-edf9028dbfbc service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:20.098911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dc97a-7016-4778-8530-edf9028dbfbc service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:20.253410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:20.254079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:20.254372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:20.254665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] PATH_INFO: `/v3/users/b327742cfd674637a53a81a4c6e198e4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:20.262766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b327742cfd674637a53a81a4c6e198e4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:20.267768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:20.415435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:20.417037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:20.448626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c9fcb-b595-4e0a-b42b-f64ac0f19c9d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:20.463320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:20.464963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:20.466671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:20.467260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] PATH_INFO: `/v3/projects/5b1447b864ba4b2faae763b20bda8003` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:20.473778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1447b864ba4b2faae763b20bda8003)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:20.479281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:20.643448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b327742cfd674637a53a81a4c6e198e4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.644369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b327742cfd674637a53a81a4c6e198e4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.672097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b327742cfd674637a53a81a4c6e198e4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.707872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1447b864ba4b2faae763b20bda8003', 'request_id': 'req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.717543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1447b864ba4b2faae763b20bda8003', 'request_id': 'req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.723144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1447b864ba4b2faae763b20bda8003', 'request_id': 'req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.853717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b327742cfd674637a53a81a4c6e198e4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.853717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b327742cfd674637a53a81a4c6e198e4'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.886579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 840/1661] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:20 2022] DELETE /identity/v3/users/b327742cfd674637a53a81a4c6e198e4 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:20.887549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:20.893757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:20.987607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] The token cache is being invalidate because project 5b1447b864ba4b2faae763b20bda8003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:20.987912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.989091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:20.992304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 822/1662] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:20 2022] DELETE /identity/v3/projects/5b1447b864ba4b2faae763b20bda8003 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:38:20.996904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:21.000510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:21.053260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ea64-5f75-429c-9b2d-daa1d3630982 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:21.064945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:21.066188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:21.066643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:21.067068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] PATH_INFO: `/v3/projects/088b026900964e37972e536090575868` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:21.070662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=088b026900964e37972e536090575868)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:21.076138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:21.120730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:21.208809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff8fc-e871-4a21-aaf4-fcc99c2359eb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:21.221959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:21.222765 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:21.223163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:21.224064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] PATH_INFO: `/v3/users/3fa98e3909134510a837775aea25d406` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:21.226359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa98e3909134510a837775aea25d406)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:21.230499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:21.303288 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '088b026900964e37972e536090575868', 'request_id': 'req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.324134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '088b026900964e37972e536090575868', 'request_id': 'req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.415139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '088b026900964e37972e536090575868', 'request_id': 'req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.441237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] The token cache is being invalidate because project 088b026900964e37972e536090575868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:21.442032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.444639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.449701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 841/1663] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:20 2022] DELETE /identity/v3/projects/088b026900964e37972e536090575868 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:21.456048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:21.460892 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:21.556573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa98e3909134510a837775aea25d406'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.558918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa98e3909134510a837775aea25d406'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.582923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa98e3909134510a837775aea25d406'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.587752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa98e3909134510a837775aea25d406'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.656125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5a6a1-0d39-4bd0-9386-5b299f488ed0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:21.676053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:21.677096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:21.677493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:21.677880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] PATH_INFO: `/v3/users/49ec9aca9e8d47578c11dcd742bf0ae5` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:21.679594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49ec9aca9e8d47578c11dcd742bf0ae5)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:21.683227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:21.721684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa98e3909134510a837775aea25d406'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:21.875018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 823/1664] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:20 2022] DELETE /identity/v3/users/3fa98e3909134510a837775aea25d406 => generated 0 bytes in 879 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:22.019523 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49ec9aca9e8d47578c11dcd742bf0ae5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:22.024891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49ec9aca9e8d47578c11dcd742bf0ae5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:22.057666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49ec9aca9e8d47578c11dcd742bf0ae5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:22.163825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:22.207653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49ec9aca9e8d47578c11dcd742bf0ae5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:22.208335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f46258c0-b2a2-4aff-aa4d-23ca6975e8c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49ec9aca9e8d47578c11dcd742bf0ae5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:22.251777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 842/1665] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:21 2022] DELETE /identity/v3/users/49ec9aca9e8d47578c11dcd742bf0ae5 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:22.447856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:22.449493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:22.599465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8fd23-17f1-4e56-8994-30d075cf7186 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:22.724484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:22.727856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:22.728451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:22.729199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] PATH_INFO: `/v3/projects/4f94d3a505fb4a00b8e705b0e200b3f8` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:22.731747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f94d3a505fb4a00b8e705b0e200b3f8)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:22.736086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:22.960438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f94d3a505fb4a00b8e705b0e200b3f8', 'request_id': 'req-a28678ae-3303-4a6b-b4af-1f0789a31913'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:22.972812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f94d3a505fb4a00b8e705b0e200b3f8', 'request_id': 'req-a28678ae-3303-4a6b-b4af-1f0789a31913'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:22.990511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f94d3a505fb4a00b8e705b0e200b3f8', 'request_id': 'req-a28678ae-3303-4a6b-b4af-1f0789a31913'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:23.147865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] The token cache is being invalidate because project 4f94d3a505fb4a00b8e705b0e200b3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:23.152305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:23.154298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:23.162905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 824/1666] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:22 2022] DELETE /identity/v3/projects/4f94d3a505fb4a00b8e705b0e200b3f8 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:23.168425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adadd0-7347-4efe-8b92-39a29ae1f354 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:23.168923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adadd0-7347-4efe-8b92-39a29ae1f354 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:23.169009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adadd0-7347-4efe-8b92-39a29ae1f354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:23.224568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-77adadd0-7347-4efe-8b92-39a29ae1f354 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:23.226633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-77adadd0-7347-4efe-8b92-39a29ae1f354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:23.310434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:23.312248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:23.384764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 843/1667] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:23.393545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2847e6-2763-4777-b5ed-c37a48ce8750 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:23.394099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2847e6-2763-4777-b5ed-c37a48ce8750 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:23.395224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2847e6-2763-4777-b5ed-c37a48ce8750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:23.425740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-de2847e6-2763-4777-b5ed-c37a48ce8750 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:23.428227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-de2847e6-2763-4777-b5ed-c37a48ce8750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:23.459627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 844/1668] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:23.462688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2847e6-2763-4777-b5ed-c37a48ce8750 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:23.493519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2847e6-2763-4777-b5ed-c37a48ce8750 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:23.507124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28678ae-3303-4a6b-b4af-1f0789a31913 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:23.521021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:23.521671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:23.521774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:23.522510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:23.522736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:23.523021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:23.523200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:23.523456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] PATH_INFO: `/v3/users/f924d125ffa245d1a34ccdc160c7b86b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:23.524070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:23.524404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:23.526524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f924d125ffa245d1a34ccdc160c7b86b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:23.527409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:23.528597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:23.588783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 845/1669] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:23.594753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:23.613319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d22db-496c-484c-b2d1-e7f2cad71ed2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:23.626037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159db7cd-562d-41b1-8043-fcaf384c1c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:23.627815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159db7cd-562d-41b1-8043-fcaf384c1c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:23.628843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159db7cd-562d-41b1-8043-fcaf384c1c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:23.629315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159db7cd-562d-41b1-8043-fcaf384c1c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:23.630323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159db7cd-562d-41b1-8043-fcaf384c1c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:23.645969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159db7cd-562d-41b1-8043-fcaf384c1c98 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:23.765770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:23.785118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 846/1670] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:23 2022] POST /identity/v3/projects => generated 403 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:23.788370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159db7cd-562d-41b1-8043-fcaf384c1c98 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:23.940990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f924d125ffa245d1a34ccdc160c7b86b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:23.940990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f924d125ffa245d1a34ccdc160c7b86b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:23.969283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f924d125ffa245d1a34ccdc160c7b86b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:23.976307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f924d125ffa245d1a34ccdc160c7b86b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.026749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159db7cd-562d-41b1-8043-fcaf384c1c98 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:24.026749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:24.028146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:24.028725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:24.030153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:24.031596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:24.034404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:24.200924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f924d125ffa245d1a34ccdc160c7b86b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.239145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 825/1671] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:23 2022] DELETE /identity/v3/users/f924d125ffa245d1a34ccdc160c7b86b => generated 0 bytes in 930 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:24.240520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:24.242739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:24.302855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 847/1672] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:23 2022] POST /identity/v3/users => generated 302 bytes in 516 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:24.305404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:24.311189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:24.365572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a453f-1f1b-4e90-a1b5-00bdfc4f3e38 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:24.462607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d33f5-fd6b-499c-adb2-b8c4ac214db9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:24.502438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:24.503342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:24.503801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:24.504095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] PATH_INFO: `/v3/projects/4df893ed17604aa78f8367737e3b5cf5` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:24.505893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df893ed17604aa78f8367737e3b5cf5)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:24.507912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:24.547734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:24.549629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:24.549629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:24.549629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] PATH_INFO: `/v3/projects/298640606eb344698dd46b8b60e2a515` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:24.555787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=298640606eb344698dd46b8b60e2a515)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:24.556237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:24.711445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df893ed17604aa78f8367737e3b5cf5', 'request_id': 'req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.738961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df893ed17604aa78f8367737e3b5cf5', 'request_id': 'req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.755021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df893ed17604aa78f8367737e3b5cf5', 'request_id': 'req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.803918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '298640606eb344698dd46b8b60e2a515', 'request_id': 'req-ecb1e0c8-1d6b-485f-b377-db8509975269'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.823611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '298640606eb344698dd46b8b60e2a515', 'request_id': 'req-ecb1e0c8-1d6b-485f-b377-db8509975269'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.938667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] The token cache is being invalidate because project 4df893ed17604aa78f8367737e3b5cf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:24.939587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.941137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-118d7cc4-f9f7-4186-a8f5-c11a0acaecf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:24.945445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 826/1673] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:24 2022] DELETE /identity/v3/projects/4df893ed17604aa78f8367737e3b5cf5 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:24.948406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a749e-75e1-440a-9f76-36348115beab None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:24.950108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a749e-75e1-440a-9f76-36348115beab None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:24.951033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a749e-75e1-440a-9f76-36348115beab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:24.999566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3b4a749e-75e1-440a-9f76-36348115beab None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:25.002235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3b4a749e-75e1-440a-9f76-36348115beab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:25.046121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '298640606eb344698dd46b8b60e2a515', 'request_id': 'req-ecb1e0c8-1d6b-485f-b377-db8509975269'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:25.075533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] The token cache is being invalidate because project 298640606eb344698dd46b8b60e2a515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:25.077570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:25.079963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:25.085111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 848/1674] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:24 2022] DELETE /identity/v3/projects/298640606eb344698dd46b8b60e2a515 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:25.092584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:25.102134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:25.277619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:25.279502 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 827/1675] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:25.282974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3a4dc-047a-4725-96dc-f17cf2b4d391 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:25.283466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3a4dc-047a-4725-96dc-f17cf2b4d391 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:25.283964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3a4dc-047a-4725-96dc-f17cf2b4d391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:25.362982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1e0c8-1d6b-485f-b377-db8509975269 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:25.364932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cdc3a4dc-047a-4725-96dc-f17cf2b4d391 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:25.367249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cdc3a4dc-047a-4725-96dc-f17cf2b4d391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:25.374439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:25.375699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:25.376957 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:25.377504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:25.379897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:25.388082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:25.425110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 828/1676] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:25.427546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7c8a0-28d6-4eed-8a34-bf16ac3dc2d0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:25.428218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7c8a0-28d6-4eed-8a34-bf16ac3dc2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:25.428669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7c8a0-28d6-4eed-8a34-bf16ac3dc2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:25.459192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 849/1677] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:25.460362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:25.462615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:25.512930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-bdc7c8a0-28d6-4eed-8a34-bf16ac3dc2d0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:25.519739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bdc7c8a0-28d6-4eed-8a34-bf16ac3dc2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:25.588025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 829/1678] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:25.590593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372c6c-2c90-4692-9e1f-c5c4286cdb0e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:25.591201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372c6c-2c90-4692-9e1f-c5c4286cdb0e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:25.591702 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372c6c-2c90-4692-9e1f-c5c4286cdb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:25.636925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa285a-142b-42ea-b2a3-e91f070d450e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:25.663445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-23372c6c-2c90-4692-9e1f-c5c4286cdb0e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:25.665335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13355d-61e7-46c9-ae14-5b33cbd80128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:25.666040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-23372c6c-2c90-4692-9e1f-c5c4286cdb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:25.670797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13355d-61e7-46c9-ae14-5b33cbd80128 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:25.676375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13355d-61e7-46c9-ae14-5b33cbd80128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:25.678029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13355d-61e7-46c9-ae14-5b33cbd80128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:25.696022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13355d-61e7-46c9-ae14-5b33cbd80128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:25.699001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13355d-61e7-46c9-ae14-5b33cbd80128 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:25.724666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 850/1679] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:25.729009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 830/1680] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:25.729214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13355d-61e7-46c9-ae14-5b33cbd80128 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:25.729325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38919285-f33e-4728-a765-7c893ef0d376 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:25.733327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38919285-f33e-4728-a765-7c893ef0d376 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:25.733845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38919285-f33e-4728-a765-7c893ef0d376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:25.762246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13355d-61e7-46c9-ae14-5b33cbd80128 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:25.785443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e7813-8825-4ce6-a651-7618df6fc7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:25.785443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e7813-8825-4ce6-a651-7618df6fc7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:25.785443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e7813-8825-4ce6-a651-7618df6fc7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:25.785443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e7813-8825-4ce6-a651-7618df6fc7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:25.809998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e7813-8825-4ce6-a651-7618df6fc7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:25.809998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e7813-8825-4ce6-a651-7618df6fc7e7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:25.855777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 851/1681] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:25.868115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e7813-8825-4ce6-a651-7618df6fc7e7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:25.895273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e7813-8825-4ce6-a651-7618df6fc7e7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:25.911273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:25.911934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:25.912246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:25.912575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:25.914883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-38919285-f33e-4728-a765-7c893ef0d376 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:25.953676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-38919285-f33e-4728-a765-7c893ef0d376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:25.955185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:25.955185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:25.955185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:25.982454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 831/1682] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:25.992574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 852/1683] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:25.993729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:26.028001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38919285-f33e-4728-a765-7c893ef0d376 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:26.042761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0550d-9798-406c-a918-5e39bf38e73f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:26.062396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:26.068080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:26.068080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:26.068080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:26.068080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:26.068080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:26.068080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:26.084636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38919285-f33e-4728-a765-7c893ef0d376 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:26.113517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 853/1684] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:26.113517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:26.137751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0664-b765-47ac-b1be-63085e397d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:26.145012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0664-b765-47ac-b1be-63085e397d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:26.145012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0664-b765-47ac-b1be-63085e397d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:26.146351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c0664-b765-47ac-b1be-63085e397d80 admin admin] PATH_INFO: `/v3/projects/68d03258876d46f1a33641921e33d0f7/users/34b156c2388640da8859b9984965b443/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:26.174521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d8334-b11f-4028-b770-56931e72e49d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:26.199337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:26.203767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:26.205980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:26.205980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:26.214483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:26.219535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:26.225747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c0664-b765-47ac-b1be-63085e397d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d03258876d46f1a33641921e33d0f7, user_id=34b156c2388640da8859b9984965b443, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:26.230993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c0664-b765-47ac-b1be-63085e397d80 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:26.371421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 832/1685] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:25 2022] PUT /identity/v3/projects/68d03258876d46f1a33641921e33d0f7/users/34b156c2388640da8859b9984965b443/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:26.373267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0664-b765-47ac-b1be-63085e397d80 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:26.395664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 854/1686] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:26 2022] POST /identity/v3/projects => generated 383 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:26.402075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:26.411572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:26.562024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:26.610849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c0664-b765-47ac-b1be-63085e397d80 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:26.618249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039584da-5450-478a-842b-2fcf43b439a1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:26.621087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd818d-b712-42a1-af17-265d2131f0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:26.623009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd818d-b712-42a1-af17-265d2131f0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:26.623009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd818d-b712-42a1-af17-265d2131f0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:26.623361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd818d-b712-42a1-af17-265d2131f0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:26.623938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd818d-b712-42a1-af17-265d2131f0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:26.627055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd818d-b712-42a1-af17-265d2131f0a3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:26.633513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:26.634196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:26.634488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:26.634769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] PATH_INFO: `/v3/users/fce6a5fc556c4cec9fbe79960cce25ae` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:26.637097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce6a5fc556c4cec9fbe79960cce25ae)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:26.645396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:26.799841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 833/1687] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:26 2022] POST /identity/v3/projects => generated 357 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:26.808411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f714-135a-4c56-8642-fa3cbf64196c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:26.808411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f714-135a-4c56-8642-fa3cbf64196c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:26.808411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f714-135a-4c56-8642-fa3cbf64196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:26.878661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3460f714-135a-4c56-8642-fa3cbf64196c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:26.879953 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3460f714-135a-4c56-8642-fa3cbf64196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:26.968380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce6a5fc556c4cec9fbe79960cce25ae'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:26.969931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce6a5fc556c4cec9fbe79960cce25ae'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:27.007987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce6a5fc556c4cec9fbe79960cce25ae'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:27.092649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 834/1688] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:27.093559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce6a5fc556c4cec9fbe79960cce25ae'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:27.094250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce6a5fc556c4cec9fbe79960cce25ae'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:27.094416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460f714-135a-4c56-8642-fa3cbf64196c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:27.100700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3460f714-135a-4c56-8642-fa3cbf64196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:27.142516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460f714-135a-4c56-8642-fa3cbf64196c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:27.142516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494aac39-2431-4652-8dbd-b93a795eb32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:27.142516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aac39-2431-4652-8dbd-b93a795eb32c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:27.152770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aac39-2431-4652-8dbd-b93a795eb32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:27.155981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494aac39-2431-4652-8dbd-b93a795eb32c admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:27.161648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494aac39-2431-4652-8dbd-b93a795eb32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:27.174529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:27.180224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 855/1689] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:26 2022] DELETE /identity/v3/users/fce6a5fc556c4cec9fbe79960cce25ae => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:27.185838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:27.187296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494aac39-2431-4652-8dbd-b93a795eb32c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:27.188348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:27.475576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3831fa2-11cb-4f6f-8662-6b1b344e9858 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:27.491412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:27.493572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:27.494211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:27.494923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:27.525463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:27.536647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:27.704925 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 835/1690] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:27 2022] POST /identity/v3/users => generated 292 bytes in 611 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:27.722988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4a749-cf6b-4b58-93f9-c564190ae8ef None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:27.723388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4a749-cf6b-4b58-93f9-c564190ae8ef None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:27.723666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4a749-cf6b-4b58-93f9-c564190ae8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:27.723929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 856/1691] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:27 2022] POST /identity/v3/users => generated 279 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:27.726599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:27.734060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:27.801841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-aed4a749-cf6b-4b58-93f9-c564190ae8ef None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:27.803424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-aed4a749-cf6b-4b58-93f9-c564190ae8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:27.838615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 836/1692] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:27.840401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f3dce-6b0b-4b2e-a323-ea3a4f85bcb4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:27.840869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f3dce-6b0b-4b2e-a323-ea3a4f85bcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:27.841320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f3dce-6b0b-4b2e-a323-ea3a4f85bcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:27.862223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ce620-4c02-46d8-bf38-e5e4b5135180 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:27.869449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9992df-f491-48f2-8227-eb7fd645da4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:27.871414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9992df-f491-48f2-8227-eb7fd645da4a service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:27.872843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9992df-f491-48f2-8227-eb7fd645da4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:27.873315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9992df-f491-48f2-8227-eb7fd645da4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:27.911158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9992df-f491-48f2-8227-eb7fd645da4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:27.914392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8f2f3dce-6b0b-4b2e-a323-ea3a4f85bcb4 None None] MFA Rules not processed for user `0108005591b14c5081d460f5c1a7526f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:27.917436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9992df-f491-48f2-8227-eb7fd645da4a service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:27.918972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8f2f3dce-6b0b-4b2e-a323-ea3a4f85bcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:27.957079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 857/1693] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:27.959153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9992df-f491-48f2-8227-eb7fd645da4a service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:27.983438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9992df-f491-48f2-8227-eb7fd645da4a service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:27.994630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70f62c-21ca-4300-9595-0e0e3afde979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:27.995373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70f62c-21ca-4300-9595-0e0e3afde979 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:27.995756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70f62c-21ca-4300-9595-0e0e3afde979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:27.996114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70f62c-21ca-4300-9595-0e0e3afde979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:27.996996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70f62c-21ca-4300-9595-0e0e3afde979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:27.998974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70f62c-21ca-4300-9595-0e0e3afde979 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:28.013353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 858/1694] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:28.014436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70f62c-21ca-4300-9595-0e0e3afde979 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:28.028350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70f62c-21ca-4300-9595-0e0e3afde979 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:28.034658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:28.035452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.035767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.036212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:28.036907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:28.038856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:28.072592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 859/1695] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:28.072592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:28.072592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:28.096074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b615a-2cc1-4ab3-a970-9e4673cffc44 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:28.101461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 837/1696] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:28.102257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf84f-0fac-4450-9efc-568cd1752210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:28.102414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f3dce-6b0b-4b2e-a323-ea3a4f85bcb4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:28.102547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf84f-0fac-4450-9efc-568cd1752210 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.102684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf84f-0fac-4450-9efc-568cd1752210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.103063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf84f-0fac-4450-9efc-568cd1752210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:28.116336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbf84f-0fac-4450-9efc-568cd1752210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:28.118224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbf84f-0fac-4450-9efc-568cd1752210 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:28.132233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f3dce-6b0b-4b2e-a323-ea3a4f85bcb4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:28.132486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 860/1697] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:28.133311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf84f-0fac-4450-9efc-568cd1752210 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:28.147419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf84f-0fac-4450-9efc-568cd1752210 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:28.149104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2ddf2-1f4a-45a4-92f5-01cbfc7807c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:28.149836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2ddf2-1f4a-45a4-92f5-01cbfc7807c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.150123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2ddf2-1f4a-45a4-92f5-01cbfc7807c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.150401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2ddf2-1f4a-45a4-92f5-01cbfc7807c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:28.152043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2ddf2-1f4a-45a4-92f5-01cbfc7807c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:28.153826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78daec-7808-4dbc-970b-ae63467b5924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:28.154472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78daec-7808-4dbc-970b-ae63467b5924 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.154830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78daec-7808-4dbc-970b-ae63467b5924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.155181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78daec-7808-4dbc-970b-ae63467b5924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:28.156014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78daec-7808-4dbc-970b-ae63467b5924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:28.156966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2ddf2-1f4a-45a4-92f5-01cbfc7807c6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:28.158441 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78daec-7808-4dbc-970b-ae63467b5924 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:28.176490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 861/1698] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:28.177963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568e274-5c81-4eea-980f-fba1e67f0bab None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.178332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568e274-5c81-4eea-980f-fba1e67f0bab None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.178683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568e274-5c81-4eea-980f-fba1e67f0bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:28.188155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 838/1699] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:28.189746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2ddf2-1f4a-45a4-92f5-01cbfc7807c6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:28.203020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2ddf2-1f4a-45a4-92f5-01cbfc7807c6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:28.205182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7568e274-5c81-4eea-980f-fba1e67f0bab None None] MFA Rules not processed for user `0108005591b14c5081d460f5c1a7526f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:28.206690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7568e274-5c81-4eea-980f-fba1e67f0bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:28.215894 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 862/1700] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:28.216617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568e274-5c81-4eea-980f-fba1e67f0bab None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:28.226542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953dfff8-71a7-43bf-ae1f-66e1e9453d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:28.227745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dfff8-71a7-43bf-ae1f-66e1e9453d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.227745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dfff8-71a7-43bf-ae1f-66e1e9453d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.227745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953dfff8-71a7-43bf-ae1f-66e1e9453d1e admin admin] PATH_INFO: `/v3/projects/d4e3aa40c9774cdb92f3aac19c24e2bf/users/6972ec5f8d5a44a3bdaef9230d64c9cc/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:28.248067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:28.258447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568e274-5c81-4eea-980f-fba1e67f0bab None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:28.265253 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953dfff8-71a7-43bf-ae1f-66e1e9453d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e3aa40c9774cdb92f3aac19c24e2bf, user_id=6972ec5f8d5a44a3bdaef9230d64c9cc, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:28.269081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:28.269382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953dfff8-71a7-43bf-ae1f-66e1e9453d1e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:28.269783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.270072 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.270356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] PATH_INFO: `/v3/projects/47326d6dd1004f24aabd48c2db58d911/users/f223fd86ce04426c90b5699691071b88/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:28.297029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47326d6dd1004f24aabd48c2db58d911, user_id=f223fd86ce04426c90b5699691071b88, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:28.302202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:28.412629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 863/1701] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:28 2022] PUT /identity/v3/projects/47326d6dd1004f24aabd48c2db58d911/users/f223fd86ce04426c90b5699691071b88/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:28.413826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:28.420683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 839/1702] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:28 2022] PUT /identity/v3/projects/d4e3aa40c9774cdb92f3aac19c24e2bf/users/6972ec5f8d5a44a3bdaef9230d64c9cc/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:28.426044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:28.438004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff9e14-7f5f-4d64-a26a-b0442278afa5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.439116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff9e14-7f5f-4d64-a26a-b0442278afa5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.439708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff9e14-7f5f-4d64-a26a-b0442278afa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:28.581836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2cff9e14-7f5f-4d64-a26a-b0442278afa5 None None] MFA Rules not processed for user `6972ec5f8d5a44a3bdaef9230d64c9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:28.590316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2cff9e14-7f5f-4d64-a26a-b0442278afa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:28.657927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bff01-b6a3-419b-9ad1-915a1495f9b1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:28.791921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:28.965883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:28.967266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:28.967662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:28.968159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:29.036391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 840/1703] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 599 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:29.037013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78909b52-f4e4-4449-806f-b56f36d499ae None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:29.037324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78909b52-f4e4-4449-806f-b56f36d499ae None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:29.038062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78909b52-f4e4-4449-806f-b56f36d499ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:29.171636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-78909b52-f4e4-4449-806f-b56f36d499ae None None] MFA Rules not processed for user `f223fd86ce04426c90b5699691071b88`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:29.172954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-78909b52-f4e4-4449-806f-b56f36d499ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:29.279715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:29.283564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:29.372418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 864/1704] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 959 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:29.375627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:29.381350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:29.633462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087c13c-33f2-4238-b9b5-b913162e5bb3 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:29.645008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 841/1705] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 610 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:29.647044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78909b52-f4e4-4449-806f-b56f36d499ae None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:29.655853 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:29.658999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:29.668513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:29.679228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] PATH_INFO: `/v3/projects/6be8ac36241e4131b3846cd43f01a3b4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:29.682846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be8ac36241e4131b3846cd43f01a3b4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:29.689802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:29.692355 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78909b52-f4e4-4449-806f-b56f36d499ae None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:29.712783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad43e3-ecf6-45e3-8866-fcfb2c28a1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:29.713877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad43e3-ecf6-45e3-8866-fcfb2c28a1df admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:29.714363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad43e3-ecf6-45e3-8866-fcfb2c28a1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:29.714804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad43e3-ecf6-45e3-8866-fcfb2c28a1df admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:29.716075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad43e3-ecf6-45e3-8866-fcfb2c28a1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:29.728445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad43e3-ecf6-45e3-8866-fcfb2c28a1df admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:29.943491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:29.945147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 842/1706] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:29 2022] POST /identity/v3/projects => generated 357 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:29.960660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad43e3-ecf6-45e3-8866-fcfb2c28a1df admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:29.994617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be8ac36241e4131b3846cd43f01a3b4', 'request_id': 'req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:30.015105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be8ac36241e4131b3846cd43f01a3b4', 'request_id': 'req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:30.120116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be8ac36241e4131b3846cd43f01a3b4', 'request_id': 'req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:30.152706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad43e3-ecf6-45e3-8866-fcfb2c28a1df admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:30.158141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] The token cache is being invalidate because project 6be8ac36241e4131b3846cd43f01a3b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:30.161037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:30.162832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1b927ee0-d0a7-4481-a8b0-c9f07bbfd1f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:30.163035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:30.164754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:30.165526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:30.166288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:30.169593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:30.176963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 865/1707] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:29 2022] DELETE /identity/v3/projects/6be8ac36241e4131b3846cd43f01a3b4 => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:30.176963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23fec5-dbba-4545-8b8c-52275f87375f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:30.176963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23fec5-dbba-4545-8b8c-52275f87375f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:30.176963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23fec5-dbba-4545-8b8c-52275f87375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:30.189605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:30.263055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-af23fec5-dbba-4545-8b8c-52275f87375f None None] MFA Rules not processed for user `773dc87736034257b3d1ae2e8f02bf23`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:30.266132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-af23fec5-dbba-4545-8b8c-52275f87375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:30.325421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 843/1708] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:29 2022] POST /identity/v3/projects => generated 381 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:30.327003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:30.331220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:30.476745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:30.482446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cd1a2-9bf8-4e4d-98e8-e3236291dbcd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:30.490915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:30.491529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:30.491939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:30.492415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:30.498471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:30.508263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:30.509924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 866/1709] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:30.511861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03c584-0b04-4dd0-a20f-a8c68c836362 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:30.512574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03c584-0b04-4dd0-a20f-a8c68c836362 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:30.513410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03c584-0b04-4dd0-a20f-a8c68c836362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:30.561715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0c03c584-0b04-4dd0-a20f-a8c68c836362 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:30.567160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0c03c584-0b04-4dd0-a20f-a8c68c836362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:30.586189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 867/1710] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:30.587781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47814a-96fe-4ece-97f7-eeada532eacc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:30.587921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47814a-96fe-4ece-97f7-eeada532eacc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:30.588172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47814a-96fe-4ece-97f7-eeada532eacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:30.648831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fd47814a-96fe-4ece-97f7-eeada532eacc None None] MFA Rules not processed for user `ea9d4eafe59c4c8180eece8ec9283e40`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:30.649913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 844/1711] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:30 2022] POST /identity/v3/users => generated 285 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:30.650161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fd47814a-96fe-4ece-97f7-eeada532eacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:30.654655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:30.664499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:30.707928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91e26f-dffe-4161-ba37-2beb187f3d19 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:30.724898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b930b78-86c8-45aa-92b9-2ddf27bec829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:30.726752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b930b78-86c8-45aa-92b9-2ddf27bec829 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:30.727275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b930b78-86c8-45aa-92b9-2ddf27bec829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:30.727759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b930b78-86c8-45aa-92b9-2ddf27bec829 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:30.730485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b930b78-86c8-45aa-92b9-2ddf27bec829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:30.741006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b930b78-86c8-45aa-92b9-2ddf27bec829 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:30.850318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 868/1712] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:30.851384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47814a-96fe-4ece-97f7-eeada532eacc None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:30.858840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fd47814a-96fe-4ece-97f7-eeada532eacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:30.906197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 845/1713] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:30 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:30.908145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394b981-65d4-44d8-b073-2a72332f22cc None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:30.908761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394b981-65d4-44d8-b073-2a72332f22cc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:30.909416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394b981-65d4-44d8-b073-2a72332f22cc None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:30.912456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 846/1714] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 13:38:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:30.916808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc76166-2ff2-4912-aedb-b1372e3f3936 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:30.917970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc76166-2ff2-4912-aedb-b1372e3f3936 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:30.918743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc76166-2ff2-4912-aedb-b1372e3f3936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:30.961524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fcc76166-2ff2-4912-aedb-b1372e3f3936 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:30.964496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fcc76166-2ff2-4912-aedb-b1372e3f3936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:30.988896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 847/1715] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:30.995961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc76166-2ff2-4912-aedb-b1372e3f3936 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:30.998694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fcc76166-2ff2-4912-aedb-b1372e3f3936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:31.023780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc76166-2ff2-4912-aedb-b1372e3f3936 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:31.041793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:31.043223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.043813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.044795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] PATH_INFO: `/v3/projects/867d7c32e6b545f49079bfcb6886db3a` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.045187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47814a-96fe-4ece-97f7-eeada532eacc None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:31.048818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=867d7c32e6b545f49079bfcb6886db3a)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:31.051809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:31.058663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb047f5-20f0-4c54-baf0-b21a50a3456a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:31.079716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb047f5-20f0-4c54-baf0-b21a50a3456a service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.079716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb047f5-20f0-4c54-baf0-b21a50a3456a service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.079716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb047f5-20f0-4c54-baf0-b21a50a3456a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.200465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb047f5-20f0-4c54-baf0-b21a50a3456a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:31.204017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb047f5-20f0-4c54-baf0-b21a50a3456a service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:31.227335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 869/1716] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:31.229019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb047f5-20f0-4c54-baf0-b21a50a3456a service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:31.253604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb047f5-20f0-4c54-baf0-b21a50a3456a service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:31.306519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867d7c32e6b545f49079bfcb6886db3a', 'request_id': 'req-00551d3c-1723-4289-a845-32f4f6489c56'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:31.324210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867d7c32e6b545f49079bfcb6886db3a', 'request_id': 'req-00551d3c-1723-4289-a845-32f4f6489c56'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:31.340474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867d7c32e6b545f49079bfcb6886db3a', 'request_id': 'req-00551d3c-1723-4289-a845-32f4f6489c56'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:31.447622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e0cec-4932-4149-aa19-9cb99237c5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:31.448812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0cec-4932-4149-aa19-9cb99237c5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.449266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0cec-4932-4149-aa19-9cb99237c5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.449748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0cec-4932-4149-aa19-9cb99237c5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.451003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e0cec-4932-4149-aa19-9cb99237c5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:31.456413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e0cec-4932-4149-aa19-9cb99237c5f3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:31.476717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 870/1717] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:31.478618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e0cec-4932-4149-aa19-9cb99237c5f3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:31.481394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] The token cache is being invalidate because project 867d7c32e6b545f49079bfcb6886db3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:31.481954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:31.486853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-00551d3c-1723-4289-a845-32f4f6489c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:31.492353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 848/1718] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:30 2022] DELETE /identity/v3/projects/867d7c32e6b545f49079bfcb6886db3a => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:31.494202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6745-28b3-4432-95a3-d5e1d080efa5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.494683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6745-28b3-4432-95a3-d5e1d080efa5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.495118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d6745-28b3-4432-95a3-d5e1d080efa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.500167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e0cec-4932-4149-aa19-9cb99237c5f3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:31.539610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:31.540326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.540606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.541333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.542243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:31.545396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:31.566067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 871/1719] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:31.566067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:31.576660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:31.586626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa1361-b132-467a-8176-44980ef99309 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:31.593752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:31.595719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:31.596604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.596868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2e6d6745-28b3-4432-95a3-d5e1d080efa5 None None] MFA Rules not processed for user `18ca375f53414475b75dae308cd19fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:31.596989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.597311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.598204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:31.598746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:31.600728 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2e6d6745-28b3-4432-95a3-d5e1d080efa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:31.601309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:31.626154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 872/1720] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:31.627518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:31.630624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:31.651411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13f3ea-6528-4e8e-bf40-ca5787851102 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:31.657311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace55fa6-7e55-421a-a306-b2a7f7a47a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:31.659693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace55fa6-7e55-421a-a306-b2a7f7a47a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.659693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace55fa6-7e55-421a-a306-b2a7f7a47a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.659693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace55fa6-7e55-421a-a306-b2a7f7a47a06 admin admin] PATH_INFO: `/v3/projects/96af120e1e0848b6877bac6cfc8c47b7/users/06ba58daf0df47a2a86489a37b91aa1c/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.681427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace55fa6-7e55-421a-a306-b2a7f7a47a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96af120e1e0848b6877bac6cfc8c47b7, user_id=06ba58daf0df47a2a86489a37b91aa1c, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:31.685423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace55fa6-7e55-421a-a306-b2a7f7a47a06 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:31.697565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 849/1721] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:38:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:31.699006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abe88b-7c69-4d4f-8cf0-ba91378670ff None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.699301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abe88b-7c69-4d4f-8cf0-ba91378670ff None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.699618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abe88b-7c69-4d4f-8cf0-ba91378670ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.724986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-08abe88b-7c69-4d4f-8cf0-ba91378670ff None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:31.726456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-08abe88b-7c69-4d4f-8cf0-ba91378670ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:31.736705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 850/1722] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:31.737726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abe88b-7c69-4d4f-8cf0-ba91378670ff None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:31.739308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-08abe88b-7c69-4d4f-8cf0-ba91378670ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:31.799304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 873/1723] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:31 2022] PUT /identity/v3/projects/96af120e1e0848b6877bac6cfc8c47b7/users/06ba58daf0df47a2a86489a37b91aa1c/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:31.805223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06f17a-e8a5-48f8-8d1e-ac08f72ba1ae None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:31.805223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06f17a-e8a5-48f8-8d1e-ac08f72ba1ae None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:31.805223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db06f17a-e8a5-48f8-8d1e-ac08f72ba1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:31.843192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:31.867478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-db06f17a-e8a5-48f8-8d1e-ac08f72ba1ae None None] MFA Rules not processed for user `34b156c2388640da8859b9984965b443`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:31.869468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-db06f17a-e8a5-48f8-8d1e-ac08f72ba1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:31.889285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abe88b-7c69-4d4f-8cf0-ba91378670ff None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:32.043218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:32.044020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:32.044435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:32.044877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:32.054675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:32.054934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 874/1724] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:32.054934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06f17a-e8a5-48f8-8d1e-ac08f72ba1ae None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:32.054934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-db06f17a-e8a5-48f8-8d1e-ac08f72ba1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:32.242077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:32.246457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:32.288426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 851/1725] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:32.295744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:32.311807 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db06f17a-e8a5-48f8-8d1e-ac08f72ba1ae None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:32.330785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:32.336116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:32.336116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:32.336494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] PATH_INFO: `/v3/projects/0edad77faec54a10bd1cebf5441653e4/users/46edd5814dbe4a17a4b7425f3a9f7642/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:32.339247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eb9c7-3488-4467-931a-fa461f1638f3 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:32.351783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b551c3-95b5-449d-9993-50c56927a35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:32.352461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b551c3-95b5-449d-9993-50c56927a35c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:32.352737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b551c3-95b5-449d-9993-50c56927a35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:32.353009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b551c3-95b5-449d-9993-50c56927a35c admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:32.353670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b551c3-95b5-449d-9993-50c56927a35c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:32.361193 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b551c3-95b5-449d-9993-50c56927a35c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:32.381509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edad77faec54a10bd1cebf5441653e4, user_id=46edd5814dbe4a17a4b7425f3a9f7642, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:32.384341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:32.545365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 875/1726] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:32 2022] PUT /identity/v3/projects/0edad77faec54a10bd1cebf5441653e4/users/46edd5814dbe4a17a4b7425f3a9f7642/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:32.546914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:32.551195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:32.583910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 852/1727] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:32 2022] POST /identity/v3/projects => generated 365 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:32.588536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce50f-e5fc-404d-adc9-ac259792cf8c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:32.588536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce50f-e5fc-404d-adc9-ac259792cf8c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:32.588536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce50f-e5fc-404d-adc9-ac259792cf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:32.661874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-abbce50f-e5fc-404d-adc9-ac259792cf8c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:32.664288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-abbce50f-e5fc-404d-adc9-ac259792cf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:32.691908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e3fd8-d1a8-4359-a840-94f314d1fa43 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:32.786742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 853/1728] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:32.786742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbce50f-e5fc-404d-adc9-ac259792cf8c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:32.801232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbce50f-e5fc-404d-adc9-ac259792cf8c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:32.808614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:32.809445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:32.811013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:32.811462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:32.813803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:32.815992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:32.838601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 854/1729] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:32.839789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:32.846154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:32.863305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:32.864221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ba3d8-f7e8-44ed-b9bf-79a6edb59ea8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:32.870671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d13e9-a1c4-40c2-b77d-ab30ca59a3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:32.871363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d13e9-a1c4-40c2-b77d-ab30ca59a3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:32.871655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d13e9-a1c4-40c2-b77d-ab30ca59a3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:32.871934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d13e9-a1c4-40c2-b77d-ab30ca59a3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:32.872641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d13e9-a1c4-40c2-b77d-ab30ca59a3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:32.875492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d13e9-a1c4-40c2-b77d-ab30ca59a3d4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:32.879681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5153f-665f-43b4-bdfe-75f19b829330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:32.880415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5153f-665f-43b4-bdfe-75f19b829330 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:32.880885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5153f-665f-43b4-bdfe-75f19b829330 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:32.881347 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5153f-665f-43b4-bdfe-75f19b829330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:32.922373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5153f-665f-43b4-bdfe-75f19b829330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:32.928000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5153f-665f-43b4-bdfe-75f19b829330 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:32.945359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 876/1730] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:32.947152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5153f-665f-43b4-bdfe-75f19b829330 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:32.970804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5153f-665f-43b4-bdfe-75f19b829330 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:32.980107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918902c-e974-4de5-aa79-b55518b38d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:32.981217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918902c-e974-4de5-aa79-b55518b38d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:32.981715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918902c-e974-4de5-aa79-b55518b38d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:32.982190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918902c-e974-4de5-aa79-b55518b38d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:32.983425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918902c-e974-4de5-aa79-b55518b38d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:32.986742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918902c-e974-4de5-aa79-b55518b38d8e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:33.009090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 877/1731] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:33.010099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918902c-e974-4de5-aa79-b55518b38d8e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:33.025142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 855/1732] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:32 2022] POST /identity/v3/projects => generated 405 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:33.025657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d13e9-a1c4-40c2-b77d-ab30ca59a3d4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:33.030095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918902c-e974-4de5-aa79-b55518b38d8e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:33.123448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d13e9-a1c4-40c2-b77d-ab30ca59a3d4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:33.132538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:33.133215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:33.133605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:33.133894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:33.134577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:33.136418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:33.137042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:33.153825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 856/1733] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:33.154710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:33.169503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8831-aacc-4b39-af0b-741cc215c989 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:33.177456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1f2f-a34e-4cba-aaf8-6a392985e425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:33.178213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1f2f-a34e-4cba-aaf8-6a392985e425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:33.178507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1f2f-a34e-4cba-aaf8-6a392985e425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:33.178813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1f2f-a34e-4cba-aaf8-6a392985e425 admin admin] PATH_INFO: `/v3/projects/96af120e1e0848b6877bac6cfc8c47b7/users/06ba58daf0df47a2a86489a37b91aa1c/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:33.182264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff1f2f-a34e-4cba-aaf8-6a392985e425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96af120e1e0848b6877bac6cfc8c47b7, user_id=06ba58daf0df47a2a86489a37b91aa1c, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:33.186299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff1f2f-a34e-4cba-aaf8-6a392985e425 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:33.212425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 857/1734] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:33 2022] PUT /identity/v3/projects/96af120e1e0848b6877bac6cfc8c47b7/users/06ba58daf0df47a2a86489a37b91aa1c/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:33.212425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1f2f-a34e-4cba-aaf8-6a392985e425 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:33.212703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c150b-c828-4f43-ae58-42b43e036775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:33.212703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c150b-c828-4f43-ae58-42b43e036775 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:33.212703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c150b-c828-4f43-ae58-42b43e036775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:33.212703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c150b-c828-4f43-ae58-42b43e036775 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:33.213045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c150b-c828-4f43-ae58-42b43e036775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:33.214774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c150b-c828-4f43-ae58-42b43e036775 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:33.267846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:33.335370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1f2f-a34e-4cba-aaf8-6a392985e425 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:33.345133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce6604-8f9d-4048-a689-8c0a631436fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:33.347067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce6604-8f9d-4048-a689-8c0a631436fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:33.347203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce6604-8f9d-4048-a689-8c0a631436fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:33.347591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce6604-8f9d-4048-a689-8c0a631436fc admin admin] PATH_INFO: `/v3/projects/0edad77faec54a10bd1cebf5441653e4/users/46edd5814dbe4a17a4b7425f3a9f7642/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:33.354567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce6604-8f9d-4048-a689-8c0a631436fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edad77faec54a10bd1cebf5441653e4, user_id=46edd5814dbe4a17a4b7425f3a9f7642, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:33.355648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce6604-8f9d-4048-a689-8c0a631436fc admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:33.399897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 878/1735] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:33 2022] POST /identity/v3/users => generated 283 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:33.410407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c150b-c828-4f43-ae58-42b43e036775 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:33.412954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 858/1736] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:33 2022] PUT /identity/v3/projects/0edad77faec54a10bd1cebf5441653e4/users/46edd5814dbe4a17a4b7425f3a9f7642/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:33.414584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce6604-8f9d-4048-a689-8c0a631436fc admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:33.436009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c150b-c828-4f43-ae58-42b43e036775 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:33.603325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce6604-8f9d-4048-a689-8c0a631436fc admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:33.614095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe1b94-3f8a-43fb-accb-9017e4d7b2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:33.616403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe1b94-3f8a-43fb-accb-9017e4d7b2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:33.617044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe1b94-3f8a-43fb-accb-9017e4d7b2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:33.617779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe1b94-3f8a-43fb-accb-9017e4d7b2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:33.619079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe1b94-3f8a-43fb-accb-9017e4d7b2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:33.624460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe1b94-3f8a-43fb-accb-9017e4d7b2a4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:33.647456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d80dc-089f-44fb-8175-9df8f6451d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:33.648105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d80dc-089f-44fb-8175-9df8f6451d01 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:33.648421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d80dc-089f-44fb-8175-9df8f6451d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:33.648701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d80dc-089f-44fb-8175-9df8f6451d01 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:33.649361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d80dc-089f-44fb-8175-9df8f6451d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:33.651398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d80dc-089f-44fb-8175-9df8f6451d01 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:33.691478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:33.811920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 879/1737] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:33 2022] POST /identity/v3/users => generated 309 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:33.817152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e01779-0194-4f3c-a54b-9a4ec00e1e7f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:33.817476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e01779-0194-4f3c-a54b-9a4ec00e1e7f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:33.817713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e01779-0194-4f3c-a54b-9a4ec00e1e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:33.826159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 859/1738] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:33 2022] POST /identity/v3/projects => generated 379 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:33.827907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe1b94-3f8a-43fb-accb-9017e4d7b2a4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:33.903318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-25e01779-0194-4f3c-a54b-9a4ec00e1e7f None None] MFA Rules not processed for user `06ba58daf0df47a2a86489a37b91aa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:33.905954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-25e01779-0194-4f3c-a54b-9a4ec00e1e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:34.244750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe1b94-3f8a-43fb-accb-9017e4d7b2a4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:34.283720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936afeb-06f2-43f3-aa9a-4d73adc45f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:34.284411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936afeb-06f2-43f3-aa9a-4d73adc45f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:34.284699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936afeb-06f2-43f3-aa9a-4d73adc45f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:34.284988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936afeb-06f2-43f3-aa9a-4d73adc45f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:34.285675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936afeb-06f2-43f3-aa9a-4d73adc45f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:34.287383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936afeb-06f2-43f3-aa9a-4d73adc45f17 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:34.320146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 860/1739] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:34.321656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f602b7-971e-4b3d-b123-3af35396e71c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:34.321943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f602b7-971e-4b3d-b123-3af35396e71c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:34.324371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f602b7-971e-4b3d-b123-3af35396e71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:34.426514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a2f602b7-971e-4b3d-b123-3af35396e71c None None] MFA Rules not processed for user `46edd5814dbe4a17a4b7425f3a9f7642`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:34.431137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a2f602b7-971e-4b3d-b123-3af35396e71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:34.490214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 880/1740] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 678 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:34.496602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e01779-0194-4f3c-a54b-9a4ec00e1e7f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:34.555868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e01779-0194-4f3c-a54b-9a4ec00e1e7f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:34.593240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec87f9-bb38-4c58-98f8-d7b36a6929a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:34.595362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec87f9-bb38-4c58-98f8-d7b36a6929a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:34.598665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec87f9-bb38-4c58-98f8-d7b36a6929a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:34.602539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec87f9-bb38-4c58-98f8-d7b36a6929a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:34.613764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec87f9-bb38-4c58-98f8-d7b36a6929a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:34.613764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec87f9-bb38-4c58-98f8-d7b36a6929a9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:34.635560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 881/1741] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:34.637690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec87f9-bb38-4c58-98f8-d7b36a6929a9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:34.659488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec87f9-bb38-4c58-98f8-d7b36a6929a9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:34.668075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:34.669333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:34.670062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:34.670525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:34.671751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:34.676243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:34.794586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 861/1742] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:34.796608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f602b7-971e-4b3d-b123-3af35396e71c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:34.863923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f602b7-971e-4b3d-b123-3af35396e71c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:34.869691 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b3bb6-89b4-4d53-a691-94768d5e6bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:34.870876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b3bb6-89b4-4d53-a691-94768d5e6bab admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:34.871180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 882/1743] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:34 2022] POST /identity/v3/users => generated 290 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:34.872034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b3bb6-89b4-4d53-a691-94768d5e6bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:34.872741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:34.873043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b3bb6-89b4-4d53-a691-94768d5e6bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:34.874505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b3bb6-89b4-4d53-a691-94768d5e6bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:34.878456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b3bb6-89b4-4d53-a691-94768d5e6bab admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:34.881089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:34.911745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 862/1744] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:34.916375 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580648e2-14ee-49f3-abdf-a5ef6d126856 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:34.916947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580648e2-14ee-49f3-abdf-a5ef6d126856 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:34.917430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580648e2-14ee-49f3-abdf-a5ef6d126856 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:34.919698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 863/1745] 10.222.0.73 () {54 vars in 961 bytes} [Wed Aug 24 13:38:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:34.923518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580648e2-14ee-49f3-abdf-a5ef6d126856 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:34.961289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580648e2-14ee-49f3-abdf-a5ef6d126856 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:34.963842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7a06b-82aa-45da-bdc1-fa24923aca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:34.964583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7a06b-82aa-45da-bdc1-fa24923aca32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:34.965158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7a06b-82aa-45da-bdc1-fa24923aca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:34.965455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7a06b-82aa-45da-bdc1-fa24923aca32 admin admin] PATH_INFO: `/v3/projects/c8bdca90afe047a2a67e3a998f8ea4db/users/f76bd73b5e6b4e49a6d39c76b4268114/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:34.999353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7a06b-82aa-45da-bdc1-fa24923aca32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bdca90afe047a2a67e3a998f8ea4db, user_id=f76bd73b5e6b4e49a6d39c76b4268114, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:35.004144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7a06b-82aa-45da-bdc1-fa24923aca32 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:35.058950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460995b-384f-422e-8ba2-939eda97e343 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:35.065284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:35.065993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:35.066291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:35.066594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:35.088073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:35.088073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:35.100901 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 883/1746] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:35.101938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:35.103339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:35.116952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 864/1747] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:34 2022] PUT /identity/v3/projects/c8bdca90afe047a2a67e3a998f8ea4db/users/f76bd73b5e6b4e49a6d39c76b4268114/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:35.122217 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7a06b-82aa-45da-bdc1-fa24923aca32 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:35.134891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9027c-f953-471a-b91a-8306f83b36fe service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:35.242197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7a06b-82aa-45da-bdc1-fa24923aca32 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:35.257330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197deeb4-b8e1-4f73-ab85-5a53197866e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:35.259656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197deeb4-b8e1-4f73-ab85-5a53197866e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:35.259656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197deeb4-b8e1-4f73-ab85-5a53197866e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:35.259656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197deeb4-b8e1-4f73-ab85-5a53197866e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:35.259656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197deeb4-b8e1-4f73-ab85-5a53197866e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:35.261515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197deeb4-b8e1-4f73-ab85-5a53197866e6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:35.280392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 865/1748] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:35.282222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197deeb4-b8e1-4f73-ab85-5a53197866e6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:35.310768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197deeb4-b8e1-4f73-ab85-5a53197866e6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:35.324809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c7565-2103-4949-ad20-e529019e16ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:35.329104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7565-2103-4949-ad20-e529019e16ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:35.329549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7565-2103-4949-ad20-e529019e16ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:35.332020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7565-2103-4949-ad20-e529019e16ff admin admin] PATH_INFO: `/v3/projects/ab500bdb6c684e5b9d41dfbe6e0a2120/users/8371e564bd514c78a7aad56c58189e5e/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:35.368600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:35.377297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b748f9-7df8-4579-bfdd-9101446bfefc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:35.380687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b748f9-7df8-4579-bfdd-9101446bfefc service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:35.381211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b748f9-7df8-4579-bfdd-9101446bfefc service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:35.381449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b748f9-7df8-4579-bfdd-9101446bfefc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:35.386970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c7565-2103-4949-ad20-e529019e16ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab500bdb6c684e5b9d41dfbe6e0a2120, user_id=8371e564bd514c78a7aad56c58189e5e, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:35.399818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c7565-2103-4949-ad20-e529019e16ff admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:35.573426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 866/1749] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:35 2022] PUT /identity/v3/projects/ab500bdb6c684e5b9d41dfbe6e0a2120/users/8371e564bd514c78a7aad56c58189e5e/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:35.579215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4e3a-99ff-414b-9565-d1822356a429 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:35.579808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4e3a-99ff-414b-9565-d1822356a429 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:35.580319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b4e3a-99ff-414b-9565-d1822356a429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:35.583384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:35.597540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b748f9-7df8-4579-bfdd-9101446bfefc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:35.599438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b748f9-7df8-4579-bfdd-9101446bfefc service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:35.635787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 884/1750] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:35.638666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b748f9-7df8-4579-bfdd-9101446bfefc service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:35.662158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-575b4e3a-99ff-414b-9565-d1822356a429 None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:35.665186 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b748f9-7df8-4579-bfdd-9101446bfefc service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:35.678697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-575b4e3a-99ff-414b-9565-d1822356a429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:35.936056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e96c5-3e12-4f1e-8c8f-3b87e2053223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:35.937049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e96c5-3e12-4f1e-8c8f-3b87e2053223 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:35.938958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e96c5-3e12-4f1e-8c8f-3b87e2053223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:35.939647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e96c5-3e12-4f1e-8c8f-3b87e2053223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:35.940573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e96c5-3e12-4f1e-8c8f-3b87e2053223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:35.942763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e96c5-3e12-4f1e-8c8f-3b87e2053223 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:35.966141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 885/1751] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:35.967349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e96c5-3e12-4f1e-8c8f-3b87e2053223 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:35.990833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e96c5-3e12-4f1e-8c8f-3b87e2053223 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:36.011137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 867/1752] 10.222.0.73 () {58 vars in 1058 bytes} [Wed Aug 24 13:38:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:36.014169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922aea3-7ebf-43d1-9386-cf768e3fe340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:36.014614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b4e3a-99ff-414b-9565-d1822356a429 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:36.015337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922aea3-7ebf-43d1-9386-cf768e3fe340 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:36.016896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922aea3-7ebf-43d1-9386-cf768e3fe340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:36.019091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922aea3-7ebf-43d1-9386-cf768e3fe340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:36.020943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922aea3-7ebf-43d1-9386-cf768e3fe340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:36.038847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922aea3-7ebf-43d1-9386-cf768e3fe340 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:36.073614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 886/1753] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:36.079666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b35268-0cde-417f-bc59-f16d4e304f7b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:36.080732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b35268-0cde-417f-bc59-f16d4e304f7b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:36.081183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b35268-0cde-417f-bc59-f16d4e304f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:36.153128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b4e3a-99ff-414b-9565-d1822356a429 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:36.167598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:36.168609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:36.169125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:36.169613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:36.177056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:36.180263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-52b35268-0cde-417f-bc59-f16d4e304f7b None None] MFA Rules not processed for user `8371e564bd514c78a7aad56c58189e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:36.181557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-52b35268-0cde-417f-bc59-f16d4e304f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:36.362838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:36.366725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:36.389665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 887/1754] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:36.389665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b35268-0cde-417f-bc59-f16d4e304f7b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:36.408346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 868/1755] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:36.409796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:36.427284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b35268-0cde-417f-bc59-f16d4e304f7b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:36.429366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ade8-3b4b-4206-b475-a56f97920c0a service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:36.436283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965d37-c4dd-42a2-9192-4996a3306bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:36.437551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965d37-c4dd-42a2-9192-4996a3306bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:36.441616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965d37-c4dd-42a2-9192-4996a3306bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:36.442343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965d37-c4dd-42a2-9192-4996a3306bb5 admin admin] PATH_INFO: `/v3/projects/c8bdca90afe047a2a67e3a998f8ea4db/users/f76bd73b5e6b4e49a6d39c76b4268114/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:36.445937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c965d37-c4dd-42a2-9192-4996a3306bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bdca90afe047a2a67e3a998f8ea4db, user_id=f76bd73b5e6b4e49a6d39c76b4268114, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:36.446484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7b062-edb2-456b-8e54-b289d6d49651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:36.451020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7b062-edb2-456b-8e54-b289d6d49651 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:36.451874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c965d37-c4dd-42a2-9192-4996a3306bb5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:36.453122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7b062-edb2-456b-8e54-b289d6d49651 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:36.454596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7b062-edb2-456b-8e54-b289d6d49651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:36.461489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6be7b062-edb2-456b-8e54-b289d6d49651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:36.468764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 869/1756] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:36 2022] PUT /identity/v3/projects/c8bdca90afe047a2a67e3a998f8ea4db/users/f76bd73b5e6b4e49a6d39c76b4268114/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:36.471108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965d37-c4dd-42a2-9192-4996a3306bb5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:36.582515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965d37-c4dd-42a2-9192-4996a3306bb5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:36.590626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:36.591324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:36.591666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:36.591945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:36.595518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:36.635302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:36.645198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:36.659273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 870/1757] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:36.660156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7b062-edb2-456b-8e54-b289d6d49651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:36.660923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:36.662443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7b062-edb2-456b-8e54-b289d6d49651 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:36.714741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 888/1758] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:36.721581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:36.759301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a8839-9314-41e0-818a-436f482cd3e8 service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:36.771967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714559-aa7f-480e-847c-3ca412a1f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:36.771967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714559-aa7f-480e-847c-3ca412a1f538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:36.771967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714559-aa7f-480e-847c-3ca412a1f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:36.771967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714559-aa7f-480e-847c-3ca412a1f538 admin admin] PATH_INFO: `/v3/projects/e0849f6b20484c609379631852e76ba5/users/aa26424a3dd14f6f949e307d351284a6/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:36.792120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714559-aa7f-480e-847c-3ca412a1f538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0849f6b20484c609379631852e76ba5, user_id=aa26424a3dd14f6f949e307d351284a6, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:36.799058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714559-aa7f-480e-847c-3ca412a1f538 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:36.934010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 871/1759] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:36 2022] PUT /identity/v3/projects/e0849f6b20484c609379631852e76ba5/users/aa26424a3dd14f6f949e307d351284a6/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:36.955790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e1b6-b1ce-4e5f-a15d-7418a95136f6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:36.956406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e1b6-b1ce-4e5f-a15d-7418a95136f6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:36.959128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e1b6-b1ce-4e5f-a15d-7418a95136f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:36.963434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:37.082615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-30e8e1b6-b1ce-4e5f-a15d-7418a95136f6 None None] MFA Rules not processed for user `aa26424a3dd14f6f949e307d351284a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:37.084780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-30e8e1b6-b1ce-4e5f-a15d-7418a95136f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:37.266513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714559-aa7f-480e-847c-3ca412a1f538 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:37.373308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 889/1760] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:37.375857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e1b6-b1ce-4e5f-a15d-7418a95136f6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:37.556023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714559-aa7f-480e-847c-3ca412a1f538 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:37.567141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f73023-38c6-49f0-a654-6a3e99fb0e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:37.568635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f73023-38c6-49f0-a654-6a3e99fb0e0a service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:37.568932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f73023-38c6-49f0-a654-6a3e99fb0e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:37.569223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f73023-38c6-49f0-a654-6a3e99fb0e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:37.592932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e1b6-b1ce-4e5f-a15d-7418a95136f6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:37.600405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d9c0-6062-4d42-8fcc-fac3231f555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:37.601713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d9c0-6062-4d42-8fcc-fac3231f555f admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:37.602829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d9c0-6062-4d42-8fcc-fac3231f555f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:37.603216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d9c0-6062-4d42-8fcc-fac3231f555f admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:37.604229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677d9c0-6062-4d42-8fcc-fac3231f555f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:37.606598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677d9c0-6062-4d42-8fcc-fac3231f555f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:37.715244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 890/1761] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:37 2022] POST /identity/v3/projects => generated 381 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:37.721259 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d9c0-6062-4d42-8fcc-fac3231f555f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:37.734395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f73023-38c6-49f0-a654-6a3e99fb0e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:37.737066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f73023-38c6-49f0-a654-6a3e99fb0e0a service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:37.753074 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 872/1762] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:38:37.801580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:37.889085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d9c0-6062-4d42-8fcc-fac3231f555f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:37.902443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a9eff-a2cc-4d5f-a92c-954ec135c3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:37.903745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a9eff-a2cc-4d5f-a92c-954ec135c3a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:37.904026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a9eff-a2cc-4d5f-a92c-954ec135c3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:37.904403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a9eff-a2cc-4d5f-a92c-954ec135c3a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:37.907277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a9eff-a2cc-4d5f-a92c-954ec135c3a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:37.914301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a9eff-a2cc-4d5f-a92c-954ec135c3a6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:38.025356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 891/1763] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:37 2022] POST /identity/v3/users => generated 297 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:38:38.027202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f73023-38c6-49f0-a654-6a3e99fb0e0a service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:38.035188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e186fd5-73a8-4945-bd5f-386c712955ac None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:38.035483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e186fd5-73a8-4945-bd5f-386c712955ac None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:38.035765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e186fd5-73a8-4945-bd5f-386c712955ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:38.064585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f73023-38c6-49f0-a654-6a3e99fb0e0a service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:38.065774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869231a-de27-4310-a9e1-e7fb9301c7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:38.067560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869231a-de27-4310-a9e1-e7fb9301c7ef admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:38.087069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869231a-de27-4310-a9e1-e7fb9301c7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:38.087368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869231a-de27-4310-a9e1-e7fb9301c7ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:38.090882 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869231a-de27-4310-a9e1-e7fb9301c7ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:38.091939 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9e186fd5-73a8-4945-bd5f-386c712955ac None None] MFA Rules not processed for user `773dc87736034257b3d1ae2e8f02bf23`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:38.093307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9e186fd5-73a8-4945-bd5f-386c712955ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:38.098114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869231a-de27-4310-a9e1-e7fb9301c7ef admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:38.139968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 873/1764] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:38.141771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c56a-a8ea-40b6-b0bd-a8bb994a0d3b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:38.143465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c56a-a8ea-40b6-b0bd-a8bb994a0d3b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:38.144053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1c56a-a8ea-40b6-b0bd-a8bb994a0d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:38.196317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-63a1c56a-a8ea-40b6-b0bd-a8bb994a0d3b None None] MFA Rules not processed for user `ea9d4eafe59c4c8180eece8ec9283e40`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:38.197609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-63a1c56a-a8ea-40b6-b0bd-a8bb994a0d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:38.256795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 892/1765] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:38.258208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e186fd5-73a8-4945-bd5f-386c712955ac None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:38.294639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e186fd5-73a8-4945-bd5f-386c712955ac None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:38.336430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fb61c-bab6-407c-b78a-c69629e762ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:38.336430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fb61c-bab6-407c-b78a-c69629e762ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:38.336430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fb61c-bab6-407c-b78a-c69629e762ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:38.336430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fb61c-bab6-407c-b78a-c69629e762ba admin admin] PATH_INFO: `/v3/projects/01dbf0b761cb452ba43043285ce02b4e/users/ffeed0a315974a07922d6a9f667e6b49/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:38.374027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fb61c-bab6-407c-b78a-c69629e762ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=01dbf0b761cb452ba43043285ce02b4e, user_id=ffeed0a315974a07922d6a9f667e6b49, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:38.378675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fb61c-bab6-407c-b78a-c69629e762ba admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:38.404258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 874/1766] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:38.498863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 893/1767] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:38 2022] PUT /identity/v3/projects/01dbf0b761cb452ba43043285ce02b4e/users/ffeed0a315974a07922d6a9f667e6b49/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:38.504355 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1c56a-a8ea-40b6-b0bd-a8bb994a0d3b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:38.527526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1c56a-a8ea-40b6-b0bd-a8bb994a0d3b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:38.731232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377a624-27e7-46c0-9032-39cd56f2cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:38.732430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a624-27e7-46c0-9032-39cd56f2cc12 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:38.732862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a624-27e7-46c0-9032-39cd56f2cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:38.733413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a624-27e7-46c0-9032-39cd56f2cc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:38.734686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377a624-27e7-46c0-9032-39cd56f2cc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:38.738093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377a624-27e7-46c0-9032-39cd56f2cc12 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:38.767250 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 875/1768] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:38.768544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fb61c-bab6-407c-b78a-c69629e762ba admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:38.775171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:38.816167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fb61c-bab6-407c-b78a-c69629e762ba admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:38.837498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ab44c-aa98-47a1-a7b1-55df0a39eeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:38.847643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ab44c-aa98-47a1-a7b1-55df0a39eeab admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:38.848119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ab44c-aa98-47a1-a7b1-55df0a39eeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:38.851297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ab44c-aa98-47a1-a7b1-55df0a39eeab admin admin] PATH_INFO: `/v3/projects/01dbf0b761cb452ba43043285ce02b4e/users/ffeed0a315974a07922d6a9f667e6b49/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:38.869730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ab44c-aa98-47a1-a7b1-55df0a39eeab admin admin] RBAC: Authorizing `identity:create_grant(project_id=01dbf0b761cb452ba43043285ce02b4e, user_id=ffeed0a315974a07922d6a9f667e6b49, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:38.873411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ab44c-aa98-47a1-a7b1-55df0a39eeab admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:38.902222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 894/1769] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:38 2022] PUT /identity/v3/projects/01dbf0b761cb452ba43043285ce02b4e/users/ffeed0a315974a07922d6a9f667e6b49/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:38:38.920359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cf6a2-059d-4c0d-9dd6-0bbbd851d21c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:38.927073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cf6a2-059d-4c0d-9dd6-0bbbd851d21c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:38.927073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cf6a2-059d-4c0d-9dd6-0bbbd851d21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:38.989432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-088cf6a2-059d-4c0d-9dd6-0bbbd851d21c None None] MFA Rules not processed for user `ffeed0a315974a07922d6a9f667e6b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:38.992480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-088cf6a2-059d-4c0d-9dd6-0bbbd851d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:39.367698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 876/1770] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:39.387477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ab44c-aa98-47a1-a7b1-55df0a39eeab admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:39.542643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ab44c-aa98-47a1-a7b1-55df0a39eeab admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:39.554103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60b07-4506-4399-ba72-62b0af31fb28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:39.555212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60b07-4506-4399-ba72-62b0af31fb28 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:39.555743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60b07-4506-4399-ba72-62b0af31fb28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:39.556252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60b07-4506-4399-ba72-62b0af31fb28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:39.583893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd60b07-4506-4399-ba72-62b0af31fb28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:39.590189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd60b07-4506-4399-ba72-62b0af31fb28 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:39.615234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 895/1771] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:39.800468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29980de6-1ae1-4c5c-adba-b6ff835bba01 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:39.800468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29980de6-1ae1-4c5c-adba-b6ff835bba01 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:39.800468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29980de6-1ae1-4c5c-adba-b6ff835bba01 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:39.800468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 877/1772] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 13:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 13:38:39.808424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773137f0-ce38-45db-b4ea-8f0bc8422cc0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:39.816662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773137f0-ce38-45db-b4ea-8f0bc8422cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:39.816662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773137f0-ce38-45db-b4ea-8f0bc8422cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:39.843792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-773137f0-ce38-45db-b4ea-8f0bc8422cc0 None None] MFA Rules not processed for user `18ca375f53414475b75dae308cd19fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:39.850206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-773137f0-ce38-45db-b4ea-8f0bc8422cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:39.932525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 896/1773] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:38:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:39.948260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29980de6-1ae1-4c5c-adba-b6ff835bba01 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:39.961973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29980de6-1ae1-4c5c-adba-b6ff835bba01 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:39.966488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c549ae-7a20-4f69-9895-6df18f24c9ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:39.967116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c549ae-7a20-4f69-9895-6df18f24c9ee service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:39.967396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c549ae-7a20-4f69-9895-6df18f24c9ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:39.967664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c549ae-7a20-4f69-9895-6df18f24c9ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:39.983556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c549ae-7a20-4f69-9895-6df18f24c9ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:39.983556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c549ae-7a20-4f69-9895-6df18f24c9ee service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:40.002553 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 878/1774] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:38:40.394915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:40.767396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1db1e5-6b36-4e1d-a9d5-f0924d6d1362 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:40.767799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1db1e5-6b36-4e1d-a9d5-f0924d6d1362 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:40.768047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1db1e5-6b36-4e1d-a9d5-f0924d6d1362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:40.917191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bc1db1e5-6b36-4e1d-a9d5-f0924d6d1362 None None] MFA Rules not processed for user `f76bd73b5e6b4e49a6d39c76b4268114`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:40.937360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc1db1e5-6b36-4e1d-a9d5-f0924d6d1362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:41.209296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 897/1775] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:40 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:41.405538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:41.608409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c549ae-7a20-4f69-9895-6df18f24c9ee service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:41.689275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1db1e5-6b36-4e1d-a9d5-f0924d6d1362 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:41.726895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1db1e5-6b36-4e1d-a9d5-f0924d6d1362 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:41.770922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffeed0a315974a07922d6a9f667e6b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01dbf0b761cb452ba43043285ce02b4e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:41.774037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:41.775866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:41.776572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:41.778032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:41.778186 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:41.779874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:41.857405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 898/1776] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:41.870126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:41.886019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c549ae-7a20-4f69-9895-6df18f24c9ee service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:41.893437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180387bc-3923-4145-a38a-98eabe3aed09 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:41.894329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:41.895121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:41.897146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:41.898112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] PATH_INFO: `/v3/users/f223fd86ce04426c90b5699691071b88` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:41.909190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26f163-0e05-41fe-937d-3a2578fdc4f7 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffeed0a315974a07922d6a9f667e6b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01dbf0b761cb452ba43043285ce02b4e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:41.909532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f223fd86ce04426c90b5699691071b88)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:41.912521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26f163-0e05-41fe-937d-3a2578fdc4f7 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:41.914148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:41.916022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26f163-0e05-41fe-937d-3a2578fdc4f7 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:41.916837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26f163-0e05-41fe-937d-3a2578fdc4f7 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:41.922136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26f163-0e05-41fe-937d-3a2578fdc4f7 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:41.931004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26f163-0e05-41fe-937d-3a2578fdc4f7 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:42.131730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 899/1777] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:41 2022] POST /identity/v3/projects => generated 354 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:42.133265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26f163-0e05-41fe-937d-3a2578fdc4f7 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:42.233317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26f163-0e05-41fe-937d-3a2578fdc4f7 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:42.237383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffeed0a315974a07922d6a9f667e6b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01dbf0b761cb452ba43043285ce02b4e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:42.238234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:42.238547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:42.238994 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:42.240128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:42.240483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:42.242305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:42.258635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 900/1778] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:42.303493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:42.319355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c99d2-0c38-4c20-a188-f0a2a0796015 tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:42.324696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffeed0a315974a07922d6a9f667e6b49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01dbf0b761cb452ba43043285ce02b4e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:42.325384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:42.325683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:42.325968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] PATH_INFO: `/v3/projects/9e8ab36f4b5b4ce8a1828f65f06e0ed6` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:42.327153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9e8ab36f4b5b4ce8a1828f65f06e0ed6)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:42.329742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:42.436458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f223fd86ce04426c90b5699691071b88'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.437474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f223fd86ce04426c90b5699691071b88'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.465559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f223fd86ce04426c90b5699691071b88'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.465700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f223fd86ce04426c90b5699691071b88'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.578061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8ab36f4b5b4ce8a1828f65f06e0ed6', 'request_id': 'req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.584333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8ab36f4b5b4ce8a1828f65f06e0ed6', 'request_id': 'req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.609091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e2b93370-bd74-47b1-9ac5-19c2ccc27462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f223fd86ce04426c90b5699691071b88'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.634006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 879/1779] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:41 2022] DELETE /identity/v3/users/f223fd86ce04426c90b5699691071b88 => generated 0 bytes in 1026 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:42.638356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b87ed-3ffb-4933-8402-0e459095ebca None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:42.638867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b87ed-3ffb-4933-8402-0e459095ebca None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:42.639505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b87ed-3ffb-4933-8402-0e459095ebca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:42.678509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c13b87ed-3ffb-4933-8402-0e459095ebca None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:42.679912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c13b87ed-3ffb-4933-8402-0e459095ebca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:42.765384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8ab36f4b5b4ce8a1828f65f06e0ed6', 'request_id': 'req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.825231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] The token cache is being invalidate because project 9e8ab36f4b5b4ce8a1828f65f06e0ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:42.835326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 880/1780] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:42.837256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56306dcc-8843-4ad8-8de4-1a006792b34b None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:42.837989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56306dcc-8843-4ad8-8de4-1a006792b34b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:42.838682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56306dcc-8843-4ad8-8de4-1a006792b34b None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:42.838788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.839997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:42.843823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 881/1781] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:38:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:42.847349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 901/1782] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:42 2022] DELETE /identity/v3/projects/9e8ab36f4b5b4ce8a1828f65f06e0ed6 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:38:42.920576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:42.920744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669745ad-8b17-4e85-8e37-34229a61127f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:42.920974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669745ad-8b17-4e85-8e37-34229a61127f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:42.921254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669745ad-8b17-4e85-8e37-34229a61127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:42.922686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:42.968975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-669745ad-8b17-4e85-8e37-34229a61127f None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:42.970934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-669745ad-8b17-4e85-8e37-34229a61127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:43.023300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db3b05-28cf-4ee1-92cd-5a73fecc40bf tempest-VolumeQuotaClassesTest-1831134175 tempest-VolumeQuotaClassesTest-1831134175-project-admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:43.139723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 882/1783] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:38:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:43.194741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669745ad-8b17-4e85-8e37-34229a61127f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:43.195054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8173f-1721-4421-abc8-6ec3cbd7f977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:43.196832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8173f-1721-4421-abc8-6ec3cbd7f977 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:43.197223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8173f-1721-4421-abc8-6ec3cbd7f977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:43.197579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8173f-1721-4421-abc8-6ec3cbd7f977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:43.197842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-669745ad-8b17-4e85-8e37-34229a61127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:43.210060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-83f8173f-1721-4421-abc8-6ec3cbd7f977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:43.232814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669745ad-8b17-4e85-8e37-34229a61127f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:43.245098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:43.246181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:43.246888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:43.247421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:43.258803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8173f-1721-4421-abc8-6ec3cbd7f977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:43.263806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8173f-1721-4421-abc8-6ec3cbd7f977 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:43.267359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:43.271950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:43.315300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:43.327066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 902/1784] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:38:43.352796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 883/1785] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:43.735714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9492-d6b7-431c-837d-55b78cea63d4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:43.736003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9492-d6b7-431c-837d-55b78cea63d4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:43.737238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c9492-d6b7-431c-837d-55b78cea63d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:43.824413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-534c9492-d6b7-431c-837d-55b78cea63d4 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:43.824413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-534c9492-d6b7-431c-837d-55b78cea63d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:44.038966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 903/1786] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:38:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:44.239477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:44.250938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:44.266845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:44.304769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd530c-948d-4069-935d-8e08e67d4d11 service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:44.314283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:44.316056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:44.316572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:44.317137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] PATH_INFO: `/v3/users/aa26424a3dd14f6f949e307d351284a6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:44.319070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa26424a3dd14f6f949e307d351284a6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:44.325523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:44.706758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c9492-d6b7-431c-837d-55b78cea63d4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:44.706758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-534c9492-d6b7-431c-837d-55b78cea63d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:44.735510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa26424a3dd14f6f949e307d351284a6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:44.738419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa26424a3dd14f6f949e307d351284a6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:44.774103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa26424a3dd14f6f949e307d351284a6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:44.776132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa26424a3dd14f6f949e307d351284a6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:44.881515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-802d39a7-8e7d-4197-9c54-81ee3564d1ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa26424a3dd14f6f949e307d351284a6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:44.922130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 884/1787] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:44 2022] DELETE /identity/v3/users/aa26424a3dd14f6f949e307d351284a6 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:44.930232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76809b2-02a4-4916-9301-17da4b582521 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:44.930955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76809b2-02a4-4916-9301-17da4b582521 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:44.931417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76809b2-02a4-4916-9301-17da4b582521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:44.964264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c76809b2-02a4-4916-9301-17da4b582521 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:44.967329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c76809b2-02a4-4916-9301-17da4b582521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:44.973519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c9492-d6b7-431c-837d-55b78cea63d4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:45.005222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:45.057756 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 885/1788] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:45.069787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76809b2-02a4-4916-9301-17da4b582521 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:45.075757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c76809b2-02a4-4916-9301-17da4b582521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:45.087693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:45.089005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:45.089291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:45.089734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:45.092167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:45.107669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:45.259655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76809b2-02a4-4916-9301-17da4b582521 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:45.284866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:45.286726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:45.287509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:45.287805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:45.288556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:45.291112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:45.304053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 904/1789] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:38:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:38:45.305164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:45.307299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:45.320125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:45.324488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:45.326429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee304e-4ebc-42ed-ab1e-707c416fc472 service swift] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:45.337867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:45.337867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:45.337867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:45.337867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] PATH_INFO: `/v3/users/34b156c2388640da8859b9984965b443` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:45.340657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b156c2388640da8859b9984965b443)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:45.347047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:45.355319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 886/1790] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:38:45.361241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:45.365907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:45.456640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99c651-8f56-49a3-9885-471aac7a1814 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:45.483744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:45.490997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:45.491624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:45.500832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] PATH_INFO: `/v3/projects/47326d6dd1004f24aabd48c2db58d911` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:45.504997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47326d6dd1004f24aabd48c2db58d911)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:45.516692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:45.841877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47326d6dd1004f24aabd48c2db58d911', 'request_id': 'req-8daa5076-58f4-492e-a1a9-55008f5cbf26'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:45.852130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47326d6dd1004f24aabd48c2db58d911', 'request_id': 'req-8daa5076-58f4-492e-a1a9-55008f5cbf26'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:45.863105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b156c2388640da8859b9984965b443'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:45.864116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b156c2388640da8859b9984965b443'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:45.865744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47326d6dd1004f24aabd48c2db58d911', 'request_id': 'req-8daa5076-58f4-492e-a1a9-55008f5cbf26'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:45.893438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b156c2388640da8859b9984965b443'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:46.037742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] The token cache is being invalidate because project 47326d6dd1004f24aabd48c2db58d911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:46.037742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:46.038625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8daa5076-58f4-492e-a1a9-55008f5cbf26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:46.043800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 887/1791] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:45 2022] DELETE /identity/v3/projects/47326d6dd1004f24aabd48c2db58d911 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:46.045267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aad1b-6d81-4f36-a878-48598a588075 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:46.051782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aad1b-6d81-4f36-a878-48598a588075 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:46.052221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aad1b-6d81-4f36-a878-48598a588075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:46.117530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:46.129460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b156c2388640da8859b9984965b443'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:46.130417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b156c2388640da8859b9984965b443'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:46.141535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9a9aad1b-6d81-4f36-a878-48598a588075 None None] MFA Rules not processed for user `18ca375f53414475b75dae308cd19fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:46.146706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a9aad1b-6d81-4f36-a878-48598a588075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:46.212035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 905/1792] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:45 2022] DELETE /identity/v3/users/34b156c2388640da8859b9984965b443 => generated 0 bytes in 904 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:46.212035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:46.224790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:46.386648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 888/1793] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 24 13:38:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:46.420360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aad1b-6d81-4f36-a878-48598a588075 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:46.423659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a9aad1b-6d81-4f36-a878-48598a588075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:46.454213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aad1b-6d81-4f36-a878-48598a588075 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:46.463357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6375-a447-47d4-90e2-b2f680f0565b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:46.490075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:46.492044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:46.493056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:46.493860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] PATH_INFO: `/v3/users/46edd5814dbe4a17a4b7425f3a9f7642` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:46.500480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46edd5814dbe4a17a4b7425f3a9f7642)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:46.508016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:46.614324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:46.615108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:46.615461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:46.615809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:46.632423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:46.639031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:46.670137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 889/1794] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:47.087571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46edd5814dbe4a17a4b7425f3a9f7642'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:47.094478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46edd5814dbe4a17a4b7425f3a9f7642'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:47.113008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46edd5814dbe4a17a4b7425f3a9f7642'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:47.281845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:47.324202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46edd5814dbe4a17a4b7425f3a9f7642'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:47.324202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46edd5814dbe4a17a4b7425f3a9f7642'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:47.428261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 906/1795] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:46 2022] DELETE /identity/v3/users/46edd5814dbe4a17a4b7425f3a9f7642 => generated 0 bytes in 1212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:47.582699 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:47.584363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:47.606848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216126f-6265-412f-8f6d-4e2c958c6d9d service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:47.688558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:47.695709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:47.901316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:47.903418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:47.903418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:47.903418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] PATH_INFO: `/v3/projects/68d03258876d46f1a33641921e33d0f7` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:47.920764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d03258876d46f1a33641921e33d0f7)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:47.927776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:48.017889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b8964-573f-4962-9638-affb724ed09e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:48.023506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:48.025133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:48.025841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:48.026490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] PATH_INFO: `/v3/projects/e0849f6b20484c609379631852e76ba5` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:48.028873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0849f6b20484c609379631852e76ba5)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:48.032375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:48.245247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d03258876d46f1a33641921e33d0f7', 'request_id': 'req-33109403-8731-49c3-990f-47a5e5259f52'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.253381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d03258876d46f1a33641921e33d0f7', 'request_id': 'req-33109403-8731-49c3-990f-47a5e5259f52'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.261880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d03258876d46f1a33641921e33d0f7', 'request_id': 'req-33109403-8731-49c3-990f-47a5e5259f52'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.356426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0849f6b20484c609379631852e76ba5', 'request_id': 'req-f030999c-744d-4a06-a181-6baf1a7c7a96'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.379992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0849f6b20484c609379631852e76ba5', 'request_id': 'req-f030999c-744d-4a06-a181-6baf1a7c7a96'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.429909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] The token cache is being invalidate because project 68d03258876d46f1a33641921e33d0f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:48.431917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.432644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.439081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 890/1796] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:47 2022] DELETE /identity/v3/projects/68d03258876d46f1a33641921e33d0f7 => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:48.440703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:48.446256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:48.552283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0849f6b20484c609379631852e76ba5', 'request_id': 'req-f030999c-744d-4a06-a181-6baf1a7c7a96'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.581706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] The token cache is being invalidate because project e0849f6b20484c609379631852e76ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:48.582284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.583116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:48.586008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 907/1797] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:47 2022] DELETE /identity/v3/projects/e0849f6b20484c609379631852e76ba5 => generated 0 bytes in 899 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:48.593255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:48.596724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:48.622251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=1,keystone:DELETE=10,keystone:SELECT=89 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:48.686008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33109403-8731-49c3-990f-47a5e5259f52 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:48.708149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030999c-744d-4a06-a181-6baf1a7c7a96 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:48.709275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:48.710060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:48.710451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:48.711946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] PATH_INFO: `/v3/users/f76bd73b5e6b4e49a6d39c76b4268114` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:48.713489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76bd73b5e6b4e49a6d39c76b4268114)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:48.726942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:48.739097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:48.742022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:48.754200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:48.756037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] PATH_INFO: `/v3/users/ffeed0a315974a07922d6a9f667e6b49` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:48.760776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffeed0a315974a07922d6a9f667e6b49)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:48.775060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:49.144394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76bd73b5e6b4e49a6d39c76b4268114'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.146170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76bd73b5e6b4e49a6d39c76b4268114'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.165938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffeed0a315974a07922d6a9f667e6b49'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.168709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76bd73b5e6b4e49a6d39c76b4268114'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.172801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76bd73b5e6b4e49a6d39c76b4268114'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.181182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffeed0a315974a07922d6a9f667e6b49'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.212408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffeed0a315974a07922d6a9f667e6b49'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.411399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e7fcc580-b15c-47dc-a6d9-8ce6de519234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76bd73b5e6b4e49a6d39c76b4268114'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.432069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:49.463429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 891/1798] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:48 2022] DELETE /identity/v3/users/f76bd73b5e6b4e49a6d39c76b4268114 => generated 0 bytes in 1021 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:49.463429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529438-2da7-4080-b96c-c347c05f23d1 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:49.463429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529438-2da7-4080-b96c-c347c05f23d1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:49.463429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529438-2da7-4080-b96c-c347c05f23d1 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:49.464507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 892/1799] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:49.488472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9529438-2da7-4080-b96c-c347c05f23d1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:49.490681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b9529438-2da7-4080-b96c-c347c05f23d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:49.518005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffeed0a315974a07922d6a9f667e6b49'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.519449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ac69f244-689e-483d-b1bd-8f6de53e4c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffeed0a315974a07922d6a9f667e6b49'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:49.619796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 908/1800] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:48 2022] DELETE /identity/v3/users/ffeed0a315974a07922d6a9f667e6b49 => generated 0 bytes in 1027 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:49.636026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bf8d-dbee-4a29-bedd-804c86859b88 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:49.638232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bf8d-dbee-4a29-bedd-804c86859b88 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:49.638680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bf8d-dbee-4a29-bedd-804c86859b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:49.677513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9529438-2da7-4080-b96c-c347c05f23d1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:49.707331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9a68bf8d-dbee-4a29-bedd-804c86859b88 None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:49.713080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9a68bf8d-dbee-4a29-bedd-804c86859b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:49.775714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:49.776763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:49.777206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:49.777877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] PATH_INFO: `/v3/projects/0edad77faec54a10bd1cebf5441653e4` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:49.779740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0edad77faec54a10bd1cebf5441653e4)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:49.783537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:49.911225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 909/1801] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:38:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:50.010610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0edad77faec54a10bd1cebf5441653e4', 'request_id': 'req-14d5466c-7ccd-417b-a2df-f56fd71caabd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:50.048975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0edad77faec54a10bd1cebf5441653e4', 'request_id': 'req-14d5466c-7ccd-417b-a2df-f56fd71caabd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:50.081798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0edad77faec54a10bd1cebf5441653e4', 'request_id': 'req-14d5466c-7ccd-417b-a2df-f56fd71caabd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:50.214186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] The token cache is being invalidate because project 0edad77faec54a10bd1cebf5441653e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:50.214186 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:50.215203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-14d5466c-7ccd-417b-a2df-f56fd71caabd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:50.217680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 893/1802] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:49 2022] DELETE /identity/v3/projects/0edad77faec54a10bd1cebf5441653e4 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:38:50.224443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642915f9-dfa1-4133-94b4-906884fa26b6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:50.225138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642915f9-dfa1-4133-94b4-906884fa26b6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:50.225729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642915f9-dfa1-4133-94b4-906884fa26b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:50.281862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-642915f9-dfa1-4133-94b4-906884fa26b6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:50.283387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-642915f9-dfa1-4133-94b4-906884fa26b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:50.403082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 910/1803] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:50.407526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2957a-e4db-4045-becc-08b8f2f60d4c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:50.407826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2957a-e4db-4045-becc-08b8f2f60d4c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:50.408114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2957a-e4db-4045-becc-08b8f2f60d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:50.483369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d1f2957a-e4db-4045-becc-08b8f2f60d4c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:50.488347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d1f2957a-e4db-4045-becc-08b8f2f60d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:50.540519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 894/1804] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:50.544358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642915f9-dfa1-4133-94b4-906884fa26b6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:50.561068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642915f9-dfa1-4133-94b4-906884fa26b6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:50.579521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:50.582040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:50.583203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:50.583944 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:50.585024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:50.585589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:50.587921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:50.617124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2957a-e4db-4045-becc-08b8f2f60d4c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:50.618633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 911/1805] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:50.663996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2957a-e4db-4045-becc-08b8f2f60d4c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:50.711195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f48df-43df-496c-b7d5-bbd382cbfdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:50.716855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f48df-43df-496c-b7d5-bbd382cbfdc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:50.717999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f48df-43df-496c-b7d5-bbd382cbfdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:50.719957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f48df-43df-496c-b7d5-bbd382cbfdc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:50.724568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f48df-43df-496c-b7d5-bbd382cbfdc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:50.733092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f48df-43df-496c-b7d5-bbd382cbfdc8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:50.849485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:50.877543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:50.881561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:50.932039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 895/1806] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:50 2022] POST /identity/v3/projects => generated 385 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:38:50.936696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f48df-43df-496c-b7d5-bbd382cbfdc8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:51.061466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6af091-45bf-403e-b5e4-471423d1e65e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:51.115130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f48df-43df-496c-b7d5-bbd382cbfdc8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:51.130731 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:51.131374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:51.131687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:51.131982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:51.132748 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:51.135110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:51.213054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:51.213698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:51.213985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:51.214263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:51.220435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:51.255533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:51.255533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:51.297492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 912/1807] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:51.413684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 896/1808] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:50 2022] POST /identity/v3/users => generated 293 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:38:51.419924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:51.440786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:51.446039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246859d-3378-418a-83dd-e960cd9cc28e service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:51.448236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:51.454065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774830-ed5d-48ed-87ff-9bb84c0b408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:51.455006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774830-ed5d-48ed-87ff-9bb84c0b408b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:51.455636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774830-ed5d-48ed-87ff-9bb84c0b408b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:51.456768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00774830-ed5d-48ed-87ff-9bb84c0b408b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:51.457651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00774830-ed5d-48ed-87ff-9bb84c0b408b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:51.459858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00774830-ed5d-48ed-87ff-9bb84c0b408b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:51.476274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 913/1809] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:51.486118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774830-ed5d-48ed-87ff-9bb84c0b408b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:51.498632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1eb43-77f5-44aa-a21a-6bc3123a5ab8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:51.503776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00774830-ed5d-48ed-87ff-9bb84c0b408b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:51.508756 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:51.509468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:51.509751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:51.510037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] PATH_INFO: `/v3/projects/c8bdca90afe047a2a67e3a998f8ea4db` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:51.511588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635e77d-bf62-403d-8d41-68e40386d898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:51.511950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8bdca90afe047a2a67e3a998f8ea4db)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:51.513547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635e77d-bf62-403d-8d41-68e40386d898 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:51.514052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635e77d-bf62-403d-8d41-68e40386d898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:51.514474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635e77d-bf62-403d-8d41-68e40386d898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:51.515661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635e77d-bf62-403d-8d41-68e40386d898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:51.518069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635e77d-bf62-403d-8d41-68e40386d898 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:51.518566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:51.537915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 914/1810] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:51.555278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635e77d-bf62-403d-8d41-68e40386d898 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:51.599895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635e77d-bf62-403d-8d41-68e40386d898 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:51.614232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6781b-c385-4e80-be07-53c8834d97e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:51.621226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6781b-c385-4e80-be07-53c8834d97e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:51.621550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6781b-c385-4e80-be07-53c8834d97e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:51.631097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6781b-c385-4e80-be07-53c8834d97e3 admin admin] PATH_INFO: `/v3/projects/66cb7375888b430db69a4382e4f07196/users/f76693ae2afe4819b41a87e9606136fd/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:51.676382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6781b-c385-4e80-be07-53c8834d97e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cb7375888b430db69a4382e4f07196, user_id=f76693ae2afe4819b41a87e9606136fd, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:51.731100 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6781b-c385-4e80-be07-53c8834d97e3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:51.779102 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8bdca90afe047a2a67e3a998f8ea4db', 'request_id': 'req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:51.800141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8bdca90afe047a2a67e3a998f8ea4db', 'request_id': 'req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:51.827526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8bdca90afe047a2a67e3a998f8ea4db', 'request_id': 'req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:51.955485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 915/1811] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:51 2022] PUT /identity/v3/projects/66cb7375888b430db69a4382e4f07196/users/f76693ae2afe4819b41a87e9606136fd/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:51.967169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42bfc7-affa-4c20-892a-9afca4512f07 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:51.968264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42bfc7-affa-4c20-892a-9afca4512f07 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:51.969990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42bfc7-affa-4c20-892a-9afca4512f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:52.002210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] The token cache is being invalidate because project c8bdca90afe047a2a67e3a998f8ea4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:52.002922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:52.005815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3c1f3ada-df8e-4318-afe6-ec2177cd6df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:52.014352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 897/1812] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:51 2022] DELETE /identity/v3/projects/c8bdca90afe047a2a67e3a998f8ea4db => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:52.023089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f080-bca2-4a4f-8d10-848f2940f35b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:52.029137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f080-bca2-4a4f-8d10-848f2940f35b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:52.038806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66f080-bca2-4a4f-8d10-848f2940f35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:52.063144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:52.101204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ef66f080-bca2-4a4f-8d10-848f2940f35b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:52.105550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ef66f080-bca2-4a4f-8d10-848f2940f35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:52.107524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1c42bfc7-affa-4c20-892a-9afca4512f07 None None] MFA Rules not processed for user `f76693ae2afe4819b41a87e9606136fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:52.110182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1c42bfc7-affa-4c20-892a-9afca4512f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:52.299374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 898/1813] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:52.322928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd37a12-2308-4171-ab07-ca3c2838cc14 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:52.322928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd37a12-2308-4171-ab07-ca3c2838cc14 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:52.323471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 916/1814] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:52.323666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd37a12-2308-4171-ab07-ca3c2838cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:52.324586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42bfc7-affa-4c20-892a-9afca4512f07 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:52.326840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1c42bfc7-affa-4c20-892a-9afca4512f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:52.380014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1bd37a12-2308-4171-ab07-ca3c2838cc14 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:52.381175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1bd37a12-2308-4171-ab07-ca3c2838cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:52.382445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42bfc7-affa-4c20-892a-9afca4512f07 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:52.393845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 899/1815] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:52.398587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd37a12-2308-4171-ab07-ca3c2838cc14 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:52.399412 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf60e4b-fd76-4ccf-b9be-9cb72819b489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:52.400148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf60e4b-fd76-4ccf-b9be-9cb72819b489 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:52.400416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf60e4b-fd76-4ccf-b9be-9cb72819b489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:52.400690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf60e4b-fd76-4ccf-b9be-9cb72819b489 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:52.401445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf60e4b-fd76-4ccf-b9be-9cb72819b489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:52.403706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf60e4b-fd76-4ccf-b9be-9cb72819b489 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:52.430297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd37a12-2308-4171-ab07-ca3c2838cc14 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:52.492091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:52.492091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:52.492091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:52.492091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:52.493692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:52.493692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:52.493692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:52.493692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 900/1816] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:52.495062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:52.519600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31977f8-d66d-4d6a-be43-d09a5c3fbeb0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:52.529608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff9b16-47b5-4975-a105-24f9087397cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:52.533304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff9b16-47b5-4975-a105-24f9087397cb admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:52.533765 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff9b16-47b5-4975-a105-24f9087397cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:52.534366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff9b16-47b5-4975-a105-24f9087397cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:52.535703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff9b16-47b5-4975-a105-24f9087397cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:52.541523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff9b16-47b5-4975-a105-24f9087397cb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:52.891604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 917/1817] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:52 2022] POST /identity/v3/projects => generated 385 bytes in 568 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:52.903471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 901/1818] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:52 2022] POST /identity/v3/projects => generated 387 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:52.915083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf60e4b-fd76-4ccf-b9be-9cb72819b489 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:52.915220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff9b16-47b5-4975-a105-24f9087397cb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:53.043673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff9b16-47b5-4975-a105-24f9087397cb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:53.055203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:53.058323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:53.059324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:53.060525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:53.063796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:53.073249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:53.238239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf60e4b-fd76-4ccf-b9be-9cb72819b489 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:53.258128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dbf48-4cf8-4fc0-9b01-f3fa817c98e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:53.259110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dbf48-4cf8-4fc0-9b01-f3fa817c98e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:53.259763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dbf48-4cf8-4fc0-9b01-f3fa817c98e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:53.260660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845dbf48-4cf8-4fc0-9b01-f3fa817c98e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:53.266192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dbf48-4cf8-4fc0-9b01-f3fa817c98e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:53.266192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845dbf48-4cf8-4fc0-9b01-f3fa817c98e8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:53.311905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 902/1819] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:52 2022] POST /identity/v3/users => generated 294 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:53.318132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:53.325242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:53.353937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e669b7-b450-4bce-a67c-6a7a40624070 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:53.370972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:53.371964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:53.372305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:53.372567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] PATH_INFO: `/v3/projects/01dbf0b761cb452ba43043285ce02b4e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:53.374419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] RBAC: Authorizing `identity:delete_project(project_id=01dbf0b761cb452ba43043285ce02b4e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:53.388561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:53.413153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 918/1820] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:52 2022] POST /identity/v3/users => generated 299 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:53.413539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dbf48-4cf8-4fc0-9b01-f3fa817c98e8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:53.457280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845dbf48-4cf8-4fc0-9b01-f3fa817c98e8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:53.473659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f948a8-2de0-417d-95cf-a8991eca64c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:53.474370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f948a8-2de0-417d-95cf-a8991eca64c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:53.474659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f948a8-2de0-417d-95cf-a8991eca64c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:53.478422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f948a8-2de0-417d-95cf-a8991eca64c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:53.482790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f948a8-2de0-417d-95cf-a8991eca64c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:53.489637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f948a8-2de0-417d-95cf-a8991eca64c5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:53.515217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 919/1821] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:53.516706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f948a8-2de0-417d-95cf-a8991eca64c5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:53.546177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f948a8-2de0-417d-95cf-a8991eca64c5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:53.557287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6f464-2006-4b71-8071-fb652521d438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:53.558484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6f464-2006-4b71-8071-fb652521d438 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:53.558991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6f464-2006-4b71-8071-fb652521d438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:53.559463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6f464-2006-4b71-8071-fb652521d438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:53.560414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6f464-2006-4b71-8071-fb652521d438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:53.575020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6f464-2006-4b71-8071-fb652521d438 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:53.598723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 920/1822] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:53.599948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6f464-2006-4b71-8071-fb652521d438 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:53.615086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6f464-2006-4b71-8071-fb652521d438 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:53.626449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01dbf0b761cb452ba43043285ce02b4e', 'request_id': 'req-0fd08499-ace1-451f-9e83-826f1688564d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:53.659380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01dbf0b761cb452ba43043285ce02b4e', 'request_id': 'req-0fd08499-ace1-451f-9e83-826f1688564d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:53.687171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01dbf0b761cb452ba43043285ce02b4e', 'request_id': 'req-0fd08499-ace1-451f-9e83-826f1688564d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:53.825831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d8ad2-50cf-4490-8190-511d1ddf4e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:53.827180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d8ad2-50cf-4490-8190-511d1ddf4e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:53.827788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d8ad2-50cf-4490-8190-511d1ddf4e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:53.846016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d8ad2-50cf-4490-8190-511d1ddf4e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:53.848764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d8ad2-50cf-4490-8190-511d1ddf4e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:53.858427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d8ad2-50cf-4490-8190-511d1ddf4e02 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:53.902787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 921/1823] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:53.902787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d8ad2-50cf-4490-8190-511d1ddf4e02 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:53.913621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d8ad2-50cf-4490-8190-511d1ddf4e02 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:53.919243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ae631-00ad-4639-bad2-197a6c79725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:53.920029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ae631-00ad-4639-bad2-197a6c79725c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:53.921468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ae631-00ad-4639-bad2-197a6c79725c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:53.921795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ae631-00ad-4639-bad2-197a6c79725c admin admin] PATH_INFO: `/v3/projects/acccaba63ff540b09caed4d6bcc3dc06/users/6042b93345514ff4adab3d34e4f7381c/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:53.950238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ae631-00ad-4639-bad2-197a6c79725c admin admin] RBAC: Authorizing `identity:create_grant(project_id=acccaba63ff540b09caed4d6bcc3dc06, user_id=6042b93345514ff4adab3d34e4f7381c, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:53.955190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ae631-00ad-4639-bad2-197a6c79725c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:54.289926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] The token cache is being invalidate because project 01dbf0b761cb452ba43043285ce02b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:38:54.290864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:54.291587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0fd08499-ace1-451f-9e83-826f1688564d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:54.294963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 903/1824] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:38:53 2022] DELETE /identity/v3/projects/01dbf0b761cb452ba43043285ce02b4e => generated 0 bytes in 977 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:54.300240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cad68-ea51-4e18-8bcb-12da7db16d90 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:54.300240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cad68-ea51-4e18-8bcb-12da7db16d90 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:54.300240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cad68-ea51-4e18-8bcb-12da7db16d90 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:54.301959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 904/1825] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:38:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:38:54.302984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cad68-ea51-4e18-8bcb-12da7db16d90 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:54.309095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c69cad68-ea51-4e18-8bcb-12da7db16d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:54.341191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cad68-ea51-4e18-8bcb-12da7db16d90 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:54.351297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567e285-89bc-44e1-b221-a5cc3842f0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:54.352584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e285-89bc-44e1-b221-a5cc3842f0d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:54.353068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e285-89bc-44e1-b221-a5cc3842f0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:54.353507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e285-89bc-44e1-b221-a5cc3842f0d4 admin admin] PATH_INFO: `/v3/projects/f74f68f00fdc4a9e934fe1a4dcda6bc4/users/cc643e53d16c4ecfa04e0a3bf2e0ffa8/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:54.368427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:54.374200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 922/1826] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:53 2022] PUT /identity/v3/projects/acccaba63ff540b09caed4d6bcc3dc06/users/6042b93345514ff4adab3d34e4f7381c/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:54.381192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be389-62a5-4771-be9a-3177a0ff2b01 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:54.381582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be389-62a5-4771-be9a-3177a0ff2b01 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:54.382187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494be389-62a5-4771-be9a-3177a0ff2b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:54.390911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567e285-89bc-44e1-b221-a5cc3842f0d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74f68f00fdc4a9e934fe1a4dcda6bc4, user_id=cc643e53d16c4ecfa04e0a3bf2e0ffa8, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:54.398351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567e285-89bc-44e1-b221-a5cc3842f0d4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:54.438581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-494be389-62a5-4771-be9a-3177a0ff2b01 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:54.440926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-494be389-62a5-4771-be9a-3177a0ff2b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:54.541404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 905/1827] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:54 2022] PUT /identity/v3/projects/f74f68f00fdc4a9e934fe1a4dcda6bc4/users/cc643e53d16c4ecfa04e0a3bf2e0ffa8/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:54.542969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9abe66-7c83-4360-a15a-0c179b425419 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:54.543261 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9abe66-7c83-4360-a15a-0c179b425419 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:54.543536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9abe66-7c83-4360-a15a-0c179b425419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:54.557332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 923/1828] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:54.559021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494be389-62a5-4771-be9a-3177a0ff2b01 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:54.563107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-494be389-62a5-4771-be9a-3177a0ff2b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:54.578621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494be389-62a5-4771-be9a-3177a0ff2b01 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:54.582569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4c9abe66-7c83-4360-a15a-0c179b425419 None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:54.599028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4c9abe66-7c83-4360-a15a-0c179b425419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:54.760530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89a634-198f-4648-baf1-6ef05255d0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:54.761306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a634-198f-4648-baf1-6ef05255d0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:54.761593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a634-198f-4648-baf1-6ef05255d0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:54.761917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89a634-198f-4648-baf1-6ef05255d0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:54.762674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89a634-198f-4648-baf1-6ef05255d0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:54.767496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89a634-198f-4648-baf1-6ef05255d0f2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:54.794621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 906/1829] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:38:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:54.797981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 924/1830] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:54.799791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18295686-8ff9-49fa-94e8-ce09eb76a9a0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:54.800081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18295686-8ff9-49fa-94e8-ce09eb76a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:54.800385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18295686-8ff9-49fa-94e8-ce09eb76a9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:54.808487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12240cb-9f69-4fb9-9aa5-fea770cc1ba0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:54.808487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12240cb-9f69-4fb9-9aa5-fea770cc1ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:54.808487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12240cb-9f69-4fb9-9aa5-fea770cc1ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:54.866893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-18295686-8ff9-49fa-94e8-ce09eb76a9a0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:54.871287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-18295686-8ff9-49fa-94e8-ce09eb76a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:54.886137 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c12240cb-9f69-4fb9-9aa5-fea770cc1ba0 None None] MFA Rules not processed for user `cc643e53d16c4ecfa04e0a3bf2e0ffa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:54.895031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c12240cb-9f69-4fb9-9aa5-fea770cc1ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:54.905185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 925/1831] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:54.913051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18295686-8ff9-49fa-94e8-ce09eb76a9a0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:54.956588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18295686-8ff9-49fa-94e8-ce09eb76a9a0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:54.971511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:54.972993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:54.973833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:54.974126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] PATH_INFO: `/v3/projects/acccaba63ff540b09caed4d6bcc3dc06/users/6042b93345514ff4adab3d34e4f7381c/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:54.982358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acccaba63ff540b09caed4d6bcc3dc06, user_id=6042b93345514ff4adab3d34e4f7381c, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:54.993099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:55.058003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 926/1832] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:54 2022] PUT /identity/v3/projects/acccaba63ff540b09caed4d6bcc3dc06/users/6042b93345514ff4adab3d34e4f7381c/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:55.063856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:55.066505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:55.135070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 907/1833] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:55.135925 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12240cb-9f69-4fb9-9aa5-fea770cc1ba0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:55.165733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12240cb-9f69-4fb9-9aa5-fea770cc1ba0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:55.272718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c67a70-6af3-4468-b953-6da919ce7063 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:55.290816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a12be-a04e-476c-8f10-57344105598c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:55.292733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a12be-a04e-476c-8f10-57344105598c service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:55.293490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a12be-a04e-476c-8f10-57344105598c service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:55.294579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a12be-a04e-476c-8f10-57344105598c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:55.344673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee712d-3f83-41b1-84f0-4e82b9cebe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:55.534461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee712d-3f83-41b1-84f0-4e82b9cebe09 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:55.534461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee712d-3f83-41b1-84f0-4e82b9cebe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:55.534461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee712d-3f83-41b1-84f0-4e82b9cebe09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:55.534461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee712d-3f83-41b1-84f0-4e82b9cebe09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:38:55.534461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee712d-3f83-41b1-84f0-4e82b9cebe09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:55.534461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee712d-3f83-41b1-84f0-4e82b9cebe09 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:55.537326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a12be-a04e-476c-8f10-57344105598c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:55.537326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a12be-a04e-476c-8f10-57344105598c service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:55.538384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 908/1834] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:38:55.538384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a829b-48a1-4626-9342-5141445c0c7e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:55.538384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a829b-48a1-4626-9342-5141445c0c7e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:55.538384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a829b-48a1-4626-9342-5141445c0c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:55.539275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 927/1835] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:55.539275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a12be-a04e-476c-8f10-57344105598c service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:55.539275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a12be-a04e-476c-8f10-57344105598c service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:55.539275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:55.540122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:55.540122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:55.540122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:55.540122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:55.540122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:55.541369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0a1a829b-48a1-4626-9342-5141445c0c7e None None] MFA Rules not processed for user `6042b93345514ff4adab3d34e4f7381c`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:55.541369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0a1a829b-48a1-4626-9342-5141445c0c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:55.633509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 909/1836] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:55.634503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a829b-48a1-4626-9342-5141445c0c7e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:55.661496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a829b-48a1-4626-9342-5141445c0c7e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:55.674713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262378c-42fc-458b-8a81-29b82fe2d57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:55.675581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262378c-42fc-458b-8a81-29b82fe2d57e admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:55.675857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262378c-42fc-458b-8a81-29b82fe2d57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:55.676030 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262378c-42fc-458b-8a81-29b82fe2d57e admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:55.677666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262378c-42fc-458b-8a81-29b82fe2d57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:55.678627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262378c-42fc-458b-8a81-29b82fe2d57e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:55.800727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 928/1837] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:55 2022] POST /identity/v3/projects => generated 387 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:55.801689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:55.806529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:55.827545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 910/1838] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:55 2022] POST /identity/v3/projects => generated 369 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:55.827545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262378c-42fc-458b-8a81-29b82fe2d57e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:56.065692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83586964-00ed-4f80-8df7-495e6dfb416d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:56.069010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262378c-42fc-458b-8a81-29b82fe2d57e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:56.117774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012209-0618-4ec6-a998-224e177e8015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:56.120643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012209-0618-4ec6-a998-224e177e8015 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:56.123725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012209-0618-4ec6-a998-224e177e8015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:56.125421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34012209-0618-4ec6-a998-224e177e8015 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:56.131545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34012209-0618-4ec6-a998-224e177e8015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:56.148837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34012209-0618-4ec6-a998-224e177e8015 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:56.177273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b4ac6-64eb-4361-b8d3-ed827a9e0f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:56.178602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4ac6-64eb-4361-b8d3-ed827a9e0f9d service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:56.179107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4ac6-64eb-4361-b8d3-ed827a9e0f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:56.179909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4ac6-64eb-4361-b8d3-ed827a9e0f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:56.327602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b4ac6-64eb-4361-b8d3-ed827a9e0f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:56.330055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b4ac6-64eb-4361-b8d3-ed827a9e0f9d service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:56.358868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 929/1839] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:56.364285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b4ac6-64eb-4361-b8d3-ed827a9e0f9d service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:56.367189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 911/1840] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:55 2022] POST /identity/v3/users => generated 294 bytes in 540 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:56.369208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012209-0618-4ec6-a998-224e177e8015 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:56.392149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b4ac6-64eb-4361-b8d3-ed827a9e0f9d service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:56.411855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1401c37-dd4f-4d47-b6ee-af5c314959dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:56.413307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1401c37-dd4f-4d47-b6ee-af5c314959dc admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:56.413669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1401c37-dd4f-4d47-b6ee-af5c314959dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:56.414227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1401c37-dd4f-4d47-b6ee-af5c314959dc admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:56.414973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1401c37-dd4f-4d47-b6ee-af5c314959dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:56.420335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34012209-0618-4ec6-a998-224e177e8015 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:56.421082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1401c37-dd4f-4d47-b6ee-af5c314959dc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:56.445862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c1e64-e7a8-41fc-9089-c8f3a74a9026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:56.446937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c1e64-e7a8-41fc-9089-c8f3a74a9026 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:56.456994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c1e64-e7a8-41fc-9089-c8f3a74a9026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:56.457579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c1e64-e7a8-41fc-9089-c8f3a74a9026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:56.459046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c1e64-e7a8-41fc-9089-c8f3a74a9026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:56.464385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c1e64-e7a8-41fc-9089-c8f3a74a9026 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:56.520245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 912/1841] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:56.527468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c1e64-e7a8-41fc-9089-c8f3a74a9026 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:56.538139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c1e64-e7a8-41fc-9089-c8f3a74a9026 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:56.547507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b77b0-d502-4cdf-9c04-7c7d05c1a3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:56.548792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b77b0-d502-4cdf-9c04-7c7d05c1a3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:56.550716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b77b0-d502-4cdf-9c04-7c7d05c1a3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:56.551543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b77b0-d502-4cdf-9c04-7c7d05c1a3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:56.553433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b77b0-d502-4cdf-9c04-7c7d05c1a3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:56.558447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b77b0-d502-4cdf-9c04-7c7d05c1a3b8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:56.610651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 913/1842] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:56.612415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b77b0-d502-4cdf-9c04-7c7d05c1a3b8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:56.661492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b77b0-d502-4cdf-9c04-7c7d05c1a3b8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:56.667283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b4e8e-2fb8-4fcd-87d5-a5f2e8890463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:56.669872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4e8e-2fb8-4fcd-87d5-a5f2e8890463 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:56.670167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4e8e-2fb8-4fcd-87d5-a5f2e8890463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:56.670429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b4e8e-2fb8-4fcd-87d5-a5f2e8890463 admin admin] PATH_INFO: `/v3/projects/f86ba1a9d2c942699c78478aa84f946e/users/d14635311a3642a7ac9535350c4da28a/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:56.700113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 930/1843] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:56 2022] POST /identity/v3/users => generated 285 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:56.706406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1401c37-dd4f-4d47-b6ee-af5c314959dc admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:56.712219 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b4e8e-2fb8-4fcd-87d5-a5f2e8890463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86ba1a9d2c942699c78478aa84f946e, user_id=d14635311a3642a7ac9535350c4da28a, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:56.719335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b4e8e-2fb8-4fcd-87d5-a5f2e8890463 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:56.773357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1401c37-dd4f-4d47-b6ee-af5c314959dc admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:56.802453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1597c-8427-4634-9572-99f19d2b5b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:56.804125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1597c-8427-4634-9572-99f19d2b5b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:56.808345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1597c-8427-4634-9572-99f19d2b5b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:56.810659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e1597c-8427-4634-9572-99f19d2b5b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:56.814333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1597c-8427-4634-9572-99f19d2b5b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:56.823425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e1597c-8427-4634-9572-99f19d2b5b02 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:56.880382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 914/1844] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:56 2022] PUT /identity/v3/projects/f86ba1a9d2c942699c78478aa84f946e/users/d14635311a3642a7ac9535350c4da28a/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:56.890562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 931/1845] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:56.894400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead04e-f9ae-4b48-b415-d1fba5d24b6c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:56.895083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead04e-f9ae-4b48-b415-d1fba5d24b6c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:56.895709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ead04e-f9ae-4b48-b415-d1fba5d24b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:56.901449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1597c-8427-4634-9572-99f19d2b5b02 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:56.971696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e1597c-8427-4634-9572-99f19d2b5b02 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:57.032216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-68ead04e-f9ae-4b48-b415-d1fba5d24b6c None None] MFA Rules not processed for user `d14635311a3642a7ac9535350c4da28a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:57.035049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-68ead04e-f9ae-4b48-b415-d1fba5d24b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:57.070019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ad928-517d-4a94-8ba2-67f47677e94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:57.070920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ad928-517d-4a94-8ba2-67f47677e94c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:57.071297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ad928-517d-4a94-8ba2-67f47677e94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:57.071455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ad928-517d-4a94-8ba2-67f47677e94c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:57.072284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ad928-517d-4a94-8ba2-67f47677e94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:57.073928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ad928-517d-4a94-8ba2-67f47677e94c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:57.087127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 932/1846] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:57.093368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ad928-517d-4a94-8ba2-67f47677e94c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:57.111203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ad928-517d-4a94-8ba2-67f47677e94c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:57.117843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97924cbf-3a2b-4a9b-9f99-24a9da8d2905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:57.118725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924cbf-3a2b-4a9b-9f99-24a9da8d2905 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:57.119015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924cbf-3a2b-4a9b-9f99-24a9da8d2905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:57.119298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97924cbf-3a2b-4a9b-9f99-24a9da8d2905 admin admin] PATH_INFO: `/v3/projects/e996ffc67446441f9a247f4c63ff1885/users/24eba350ada540c78e27556519d25ce1/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:57.152204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97924cbf-3a2b-4a9b-9f99-24a9da8d2905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e996ffc67446441f9a247f4c63ff1885, user_id=24eba350ada540c78e27556519d25ce1, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:57.156899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97924cbf-3a2b-4a9b-9f99-24a9da8d2905 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:57.355891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 915/1847] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:57.355891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ead04e-f9ae-4b48-b415-d1fba5d24b6c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:57.398881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 933/1848] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:57 2022] PUT /identity/v3/projects/e996ffc67446441f9a247f4c63ff1885/users/24eba350ada540c78e27556519d25ce1/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:38:57.412990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122728e1-0110-4a22-8548-33a3e666d6ca None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:57.413362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122728e1-0110-4a22-8548-33a3e666d6ca None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:57.413569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122728e1-0110-4a22-8548-33a3e666d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:57.418897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ead04e-f9ae-4b48-b415-d1fba5d24b6c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:57.508522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-122728e1-0110-4a22-8548-33a3e666d6ca None None] MFA Rules not processed for user `24eba350ada540c78e27556519d25ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:57.512850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-122728e1-0110-4a22-8548-33a3e666d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:57.639554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:57.639554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:57.639554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:57.647785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:57.668227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:57.668227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:57.702701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 934/1849] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:57.704329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122728e1-0110-4a22-8548-33a3e666d6ca None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:57.711276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-122728e1-0110-4a22-8548-33a3e666d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:57.763333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122728e1-0110-4a22-8548-33a3e666d6ca None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:57.779983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:57.781208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:57.781531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:57.781810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] PATH_INFO: `/v3/users/8371e564bd514c78a7aad56c58189e5e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:57.784058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8371e564bd514c78a7aad56c58189e5e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:57.788483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:57.920929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:38:57.939682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 916/1850] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:38:57 2022] POST /identity/v3/projects => generated 387 bytes in 589 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:57.939682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:57.952258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:58.144846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9da77-b763-45d6-9afb-935278bdd40b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:58.152848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23c36b-eece-47c9-86c0-bfe4b71ea115 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5c6fcfde34c05a2fec0659dfd54a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:58.154838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23c36b-eece-47c9-86c0-bfe4b71ea115 service glance] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:58.155488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23c36b-eece-47c9-86c0-bfe4b71ea115 service glance] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:58.156510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23c36b-eece-47c9-86c0-bfe4b71ea115 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:58.159181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8371e564bd514c78a7aad56c58189e5e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:58.191380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8371e564bd514c78a7aad56c58189e5e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:58.191380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8371e564bd514c78a7aad56c58189e5e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:58.314603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23c36b-eece-47c9-86c0-bfe4b71ea115 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:58.318955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23c36b-eece-47c9-86c0-bfe4b71ea115 service glance] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:58.340578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 917/1851] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:58.342235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23c36b-eece-47c9-86c0-bfe4b71ea115 service glance] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:58.449710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23c36b-eece-47c9-86c0-bfe4b71ea115 service glance] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:58.456464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:58.457098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:58.457448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:58.457788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:58.458501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:58.460908 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:58.637036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8371e564bd514c78a7aad56c58189e5e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:58.637781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d84bbc96-b723-43be-935c-fcc8d3684746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8371e564bd514c78a7aad56c58189e5e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:38:58.653601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 918/1852] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:38:58 2022] POST /identity/v3/users => generated 300 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:38:58.654677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:58.665875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:58.684824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 935/1853] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:57 2022] DELETE /identity/v3/users/8371e564bd514c78a7aad56c58189e5e => generated 0 bytes in 977 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:38:58.704009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd52dc8-d2fd-40ba-a945-60cc96903ccb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:58.709035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd52dc8-d2fd-40ba-a945-60cc96903ccb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:58.709035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd52dc8-d2fd-40ba-a945-60cc96903ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:58.729105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05205a23-a6fc-4a00-9876-f988e1abb385 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:58.764171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5bd52dc8-d2fd-40ba-a945-60cc96903ccb None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:58.770288 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5bd52dc8-d2fd-40ba-a945-60cc96903ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:58.915739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff061473-80f4-46be-ae72-34f183ed3945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:58.917021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff061473-80f4-46be-ae72-34f183ed3945 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:58.917419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff061473-80f4-46be-ae72-34f183ed3945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:58.917585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff061473-80f4-46be-ae72-34f183ed3945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:58.918280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff061473-80f4-46be-ae72-34f183ed3945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:58.919948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff061473-80f4-46be-ae72-34f183ed3945 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:58.947961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 919/1854] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:58.954356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff061473-80f4-46be-ae72-34f183ed3945 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:58.966816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 936/1855] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:38:58.992354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff061473-80f4-46be-ae72-34f183ed3945 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:58.997290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd52dc8-d2fd-40ba-a945-60cc96903ccb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:59.003000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8913c7a-0a50-440b-86ff-51be82b442e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:59.004021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8913c7a-0a50-440b-86ff-51be82b442e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:59.006191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8913c7a-0a50-440b-86ff-51be82b442e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:59.006550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5bd52dc8-d2fd-40ba-a945-60cc96903ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:59.007343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8913c7a-0a50-440b-86ff-51be82b442e9 admin admin] PATH_INFO: `/v3/projects/d283bc60ff7647a2a5aed68fc26028c1/users/a9c707d1ec1a41c0b6a4c0466799efd0/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:59.039831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8913c7a-0a50-440b-86ff-51be82b442e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d283bc60ff7647a2a5aed68fc26028c1, user_id=a9c707d1ec1a41c0b6a4c0466799efd0, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:59.042871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8913c7a-0a50-440b-86ff-51be82b442e9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:59.125951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 920/1856] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:58 2022] PUT /identity/v3/projects/d283bc60ff7647a2a5aed68fc26028c1/users/a9c707d1ec1a41c0b6a4c0466799efd0/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:38:59.133009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8913c7a-0a50-440b-86ff-51be82b442e9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:59.174903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd52dc8-d2fd-40ba-a945-60cc96903ccb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:59.290401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8913c7a-0a50-440b-86ff-51be82b442e9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:59.298204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f826c-e765-44d8-8664-86637b43ffc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:59.298867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f826c-e765-44d8-8664-86637b43ffc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:59.299202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f826c-e765-44d8-8664-86637b43ffc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:59.299609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f826c-e765-44d8-8664-86637b43ffc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:59.300412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f826c-e765-44d8-8664-86637b43ffc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:59.302633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f826c-e765-44d8-8664-86637b43ffc7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:59.323870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 921/1857] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:38:59.324462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f826c-e765-44d8-8664-86637b43ffc7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:59.339424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f826c-e765-44d8-8664-86637b43ffc7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:59.346223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10259860-ed1a-4120-b885-f8828ffcf0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:59.347132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259860-ed1a-4120-b885-f8828ffcf0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:59.347422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259860-ed1a-4120-b885-f8828ffcf0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:59.347701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259860-ed1a-4120-b885-f8828ffcf0e8 admin admin] PATH_INFO: `/v3/projects/d283bc60ff7647a2a5aed68fc26028c1/users/a9c707d1ec1a41c0b6a4c0466799efd0/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:59.350155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10259860-ed1a-4120-b885-f8828ffcf0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d283bc60ff7647a2a5aed68fc26028c1, user_id=a9c707d1ec1a41c0b6a4c0466799efd0, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:59.353410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10259860-ed1a-4120-b885-f8828ffcf0e8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:59.357937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:59.359225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:59.359460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:59.359765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:59.380074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:59.382051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 922/1858] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:38:59 2022] PUT /identity/v3/projects/d283bc60ff7647a2a5aed68fc26028c1/users/a9c707d1ec1a41c0b6a4c0466799efd0/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:38:59.383992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:59.396450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806953a-9db5-4e00-b508-065abb3a619e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:59.397357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 937/1859] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:59.418603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806953a-9db5-4e00-b508-065abb3a619e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:59.418603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806953a-9db5-4e00-b508-065abb3a619e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:59.467037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b806953a-9db5-4e00-b508-065abb3a619e None None] MFA Rules not processed for user `a9c707d1ec1a41c0b6a4c0466799efd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:38:59.469723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b806953a-9db5-4e00-b508-065abb3a619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:59.578871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 923/1860] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:38:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:38:59.664479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:59.666536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:38:59.782838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683326e-5c9d-40be-a1e5-c3313bf005f0 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:38:59.884301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:38:59.885375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:38:59.885887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:38:59.886374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:38:59.902618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:38:59.904557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:38:59.924565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 938/1861] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:38:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:38:59.982329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806953a-9db5-4e00-b508-065abb3a619e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:38:59.982329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b806953a-9db5-4e00-b508-065abb3a619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:00.132847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806953a-9db5-4e00-b508-065abb3a619e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:00.156332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:00.158148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:00.159014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:00.164266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] PATH_INFO: `/v3/users/f76693ae2afe4819b41a87e9606136fd` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:00.164266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76693ae2afe4819b41a87e9606136fd)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:00.167254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:00.301326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:00.319561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:00.335924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:00.410121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1fb5a-8521-4ec7-b2a2-fe9fc7050059 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:00.431062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76693ae2afe4819b41a87e9606136fd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.431981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76693ae2afe4819b41a87e9606136fd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.432833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:00.434099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:00.434582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:00.435053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] PATH_INFO: `/v3/projects/ab500bdb6c684e5b9d41dfbe6e0a2120` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:00.448599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab500bdb6c684e5b9d41dfbe6e0a2120)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:00.451924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76693ae2afe4819b41a87e9606136fd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.452354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:00.453671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76693ae2afe4819b41a87e9606136fd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.576332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0cf2d955-2eaf-43d6-af2b-7d1e68a5235f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76693ae2afe4819b41a87e9606136fd'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.631303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 924/1862] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:38:59 2022] DELETE /identity/v3/users/f76693ae2afe4819b41a87e9606136fd => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:00.637620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0596dbd-d834-4527-8829-db8741a30d9a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:00.638548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0596dbd-d834-4527-8829-db8741a30d9a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:00.639001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0596dbd-d834-4527-8829-db8741a30d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:00.684423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab500bdb6c684e5b9d41dfbe6e0a2120', 'request_id': 'req-382e0c9d-b3d2-4b38-9030-316bc621f154'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.710710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab500bdb6c684e5b9d41dfbe6e0a2120', 'request_id': 'req-382e0c9d-b3d2-4b38-9030-316bc621f154'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.712674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d0596dbd-d834-4527-8829-db8741a30d9a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:00.715897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d0596dbd-d834-4527-8829-db8741a30d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:00.857312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 925/1863] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:00.871303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0596dbd-d834-4527-8829-db8741a30d9a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:00.874237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d0596dbd-d834-4527-8829-db8741a30d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:00.978897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab500bdb6c684e5b9d41dfbe6e0a2120', 'request_id': 'req-382e0c9d-b3d2-4b38-9030-316bc621f154'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.981094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0596dbd-d834-4527-8829-db8741a30d9a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:00.995982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a4f7b7-3ec6-418f-9c2a-d297435baadf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:00.997272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4f7b7-3ec6-418f-9c2a-d297435baadf service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:00.997832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] The token cache is being invalidate because project ab500bdb6c684e5b9d41dfbe6e0a2120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:00.998032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4f7b7-3ec6-418f-9c2a-d297435baadf service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:00.998325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:00.998561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4f7b7-3ec6-418f-9c2a-d297435baadf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:01.000297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-382e0c9d-b3d2-4b38-9030-316bc621f154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:01.002759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 939/1864] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:00 2022] DELETE /identity/v3/projects/ab500bdb6c684e5b9d41dfbe6e0a2120 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:01.009965 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9c03-575c-4af3-b47d-3efb914095e9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:01.010561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9c03-575c-4af3-b47d-3efb914095e9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:01.011679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c9c03-575c-4af3-b47d-3efb914095e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:01.019707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4f7b7-3ec6-418f-9c2a-d297435baadf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:01.021260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a4f7b7-3ec6-418f-9c2a-d297435baadf service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:01.036008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 926/1865] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:01.044405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7c5c9c03-575c-4af3-b47d-3efb914095e9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:01.045853 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7c5c9c03-575c-4af3-b47d-3efb914095e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:01.056214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 940/1866] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:01.085830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2089f-a8ed-49e0-a5cd-8d1a718ce63e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:01.086354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2089f-a8ed-49e0-a5cd-8d1a718ce63e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:01.086835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2089f-a8ed-49e0-a5cd-8d1a718ce63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:01.117139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f5c2089f-a8ed-49e0-a5cd-8d1a718ce63e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:01.118458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f5c2089f-a8ed-49e0-a5cd-8d1a718ce63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:01.131813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 927/1867] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:01.137674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c9c03-575c-4af3-b47d-3efb914095e9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:01.160413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c9c03-575c-4af3-b47d-3efb914095e9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:01.167302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:01.168402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:01.168903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:01.169448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:01.170636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:01.171284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:01.174142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:01.212083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 941/1868] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:01.218961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2089f-a8ed-49e0-a5cd-8d1a718ce63e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:01.279995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2089f-a8ed-49e0-a5cd-8d1a718ce63e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:01.279995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301148c3-99e6-4825-8797-3a6ca8115dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:01.279995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301148c3-99e6-4825-8797-3a6ca8115dbb admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:01.279995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301148c3-99e6-4825-8797-3a6ca8115dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:01.281139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301148c3-99e6-4825-8797-3a6ca8115dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:01.281139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301148c3-99e6-4825-8797-3a6ca8115dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:01.281139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301148c3-99e6-4825-8797-3a6ca8115dbb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:01.660645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 928/1869] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:01 2022] POST /identity/v3/projects => generated 379 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:01.665284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:01.678192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf2e6c-ef73-44e2-bb4d-390afece79e5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:01.804347 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac1786-ebd7-4e38-abf8-7cefcc0d15d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:01.805217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac1786-ebd7-4e38-abf8-7cefcc0d15d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:01.805734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac1786-ebd7-4e38-abf8-7cefcc0d15d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:01.806300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac1786-ebd7-4e38-abf8-7cefcc0d15d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:01.807197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac1786-ebd7-4e38-abf8-7cefcc0d15d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:01.810387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac1786-ebd7-4e38-abf8-7cefcc0d15d0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:01.920788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 942/1870] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:01 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:01.922931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301148c3-99e6-4825-8797-3a6ca8115dbb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:01.945590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301148c3-99e6-4825-8797-3a6ca8115dbb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:01.954958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76a47e-43a3-4e9f-a72e-edc603386b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:01.956045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76a47e-43a3-4e9f-a72e-edc603386b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:01.956590 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76a47e-43a3-4e9f-a72e-edc603386b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:01.957096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76a47e-43a3-4e9f-a72e-edc603386b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:01.958562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76a47e-43a3-4e9f-a72e-edc603386b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:01.961891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76a47e-43a3-4e9f-a72e-edc603386b07 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:02.004158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 929/1871] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:02.006360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac1786-ebd7-4e38-abf8-7cefcc0d15d0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:02.044092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac1786-ebd7-4e38-abf8-7cefcc0d15d0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:02.060365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:02.061671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:02.062608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:02.063302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:02.066224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:02.069931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:02.098161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 943/1872] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:02.098712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76a47e-43a3-4e9f-a72e-edc603386b07 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:02.153170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76a47e-43a3-4e9f-a72e-edc603386b07 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:02.162775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:02.164951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:02.165807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:02.166510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] PATH_INFO: `/v3/projects/f61d348e10bc46eba7ff80f590c14002/users/964935f2a70f49fbab0a1644fae8c4a1/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:02.202300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61d348e10bc46eba7ff80f590c14002, user_id=964935f2a70f49fbab0a1644fae8c4a1, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:02.205879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:02.299779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:02.323425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:02.355451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:02.355938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71372ee5-cacc-4de4-9d17-1cd062de76de admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:02.361563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 930/1873] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:02 2022] PUT /identity/v3/projects/f61d348e10bc46eba7ff80f590c14002/users/964935f2a70f49fbab0a1644fae8c4a1/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:02.362462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:02.367839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:02.474404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:02.478540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:02.478540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:02.479094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] PATH_INFO: `/v3/projects/66cb7375888b430db69a4382e4f07196` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:02.482367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=66cb7375888b430db69a4382e4f07196)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:02.487873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:02.496516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f66513-e1e2-4ac1-890b-c388617f68e8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:02.510454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcca76-35d6-461f-8689-e36153e85adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:02.513200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcca76-35d6-461f-8689-e36153e85adf service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:02.513713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcca76-35d6-461f-8689-e36153e85adf service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:02.514142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcca76-35d6-461f-8689-e36153e85adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:02.516929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-70bcca76-35d6-461f-8689-e36153e85adf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:02.601721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcca76-35d6-461f-8689-e36153e85adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:02.603727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcca76-35d6-461f-8689-e36153e85adf service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:02.614815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 931/1874] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:02.616126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ed38c-5bee-4bb9-9729-b03c6859260d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:02.616452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ed38c-5bee-4bb9-9729-b03c6859260d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:02.616733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ed38c-5bee-4bb9-9729-b03c6859260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:02.651012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-379ed38c-5bee-4bb9-9729-b03c6859260d None None] MFA Rules not processed for user `964935f2a70f49fbab0a1644fae8c4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:02.652411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-379ed38c-5bee-4bb9-9729-b03c6859260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:02.724804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66cb7375888b430db69a4382e4f07196', 'request_id': 'req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:02.750232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66cb7375888b430db69a4382e4f07196', 'request_id': 'req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:02.802312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 932/1875] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:02.803740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ed38c-5bee-4bb9-9729-b03c6859260d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:02.818126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ed38c-5bee-4bb9-9729-b03c6859260d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:02.884284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66cb7375888b430db69a4382e4f07196', 'request_id': 'req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:02.906193 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:02.907800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] The token cache is being invalidate because project 66cb7375888b430db69a4382e4f07196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:02.908112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:02.908949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:02.909703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:02.910262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:02.910612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 944/1876] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:02 2022] DELETE /identity/v3/projects/66cb7375888b430db69a4382e4f07196 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:02.915736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:02.916462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:02.917157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:02.918384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:02.922640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:02.936225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31f9bb-3d62-4af8-a8c7-08658b7a1f7c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:02.943535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:02.944254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:02.944537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:02.944804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] PATH_INFO: `/v3/users/6042b93345514ff4adab3d34e4f7381c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:02.946099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6042b93345514ff4adab3d34e4f7381c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:02.948387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:03.034902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 933/1877] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:02 2022] POST /identity/v3/projects => generated 381 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:03.039207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:03.040990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:03.136069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c5e79-6bdd-455d-8128-1d24f0fb64d5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:03.142919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:03.143892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:03.144220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:03.144498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:03.145173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:03.147442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:03.221805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6042b93345514ff4adab3d34e4f7381c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:03.222644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6042b93345514ff4adab3d34e4f7381c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:03.244911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6042b93345514ff4adab3d34e4f7381c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:03.254586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 934/1878] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:03 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:03.259680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:03.261520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:03.280040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcbf92-d9ce-4467-b9cd-0c77f2a8feec admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:03.286273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c0b8a-1171-4696-99be-6de0fde0c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:03.287001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c0b8a-1171-4696-99be-6de0fde0c5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:03.295759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c0b8a-1171-4696-99be-6de0fde0c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:03.295759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c0b8a-1171-4696-99be-6de0fde0c5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:03.295759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c0b8a-1171-4696-99be-6de0fde0c5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:03.295759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c0b8a-1171-4696-99be-6de0fde0c5a4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:03.303280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 935/1879] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:03.307761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c0b8a-1171-4696-99be-6de0fde0c5a4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:03.323505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c0b8a-1171-4696-99be-6de0fde0c5a4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:03.333004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc08cc-64c3-40e1-a1ad-5706560a88e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:03.334204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc08cc-64c3-40e1-a1ad-5706560a88e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:03.334714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc08cc-64c3-40e1-a1ad-5706560a88e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:03.335207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc08cc-64c3-40e1-a1ad-5706560a88e5 admin admin] PATH_INFO: `/v3/projects/81a53bef17a6484989a0e82b57e471fb/users/b8db9348ec5e4fda92118bbaf35cb21c/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:03.360665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc08cc-64c3-40e1-a1ad-5706560a88e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a53bef17a6484989a0e82b57e471fb, user_id=b8db9348ec5e4fda92118bbaf35cb21c, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:03.364959 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6042b93345514ff4adab3d34e4f7381c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:03.365944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc08cc-64c3-40e1-a1ad-5706560a88e5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:03.371353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6042b93345514ff4adab3d34e4f7381c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:03.435406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 945/1880] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:02 2022] DELETE /identity/v3/users/6042b93345514ff4adab3d34e4f7381c => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:03.483114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 936/1881] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:03 2022] PUT /identity/v3/projects/81a53bef17a6484989a0e82b57e471fb/users/b8db9348ec5e4fda92118bbaf35cb21c/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:03.493629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:03.658408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eafdd-91b7-4efe-862b-fdda7655300d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:03.787059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af9af-778e-46bd-95b1-6b71b75e96ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:03.788384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af9af-778e-46bd-95b1-6b71b75e96ae admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:03.788980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af9af-778e-46bd-95b1-6b71b75e96ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:03.789882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af9af-778e-46bd-95b1-6b71b75e96ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:03.791374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af9af-778e-46bd-95b1-6b71b75e96ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:03.796385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af9af-778e-46bd-95b1-6b71b75e96ae admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:03.818332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 946/1882] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:03.838068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc08cc-64c3-40e1-a1ad-5706560a88e5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:03.876725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc08cc-64c3-40e1-a1ad-5706560a88e5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:03.878394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:03.880280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:03.880280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:03.882111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] PATH_INFO: `/v3/projects/81a53bef17a6484989a0e82b57e471fb/users/b8db9348ec5e4fda92118bbaf35cb21c/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:03.884779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a53bef17a6484989a0e82b57e471fb, user_id=b8db9348ec5e4fda92118bbaf35cb21c, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:03.891248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:03.938110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 937/1883] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:03 2022] PUT /identity/v3/projects/81a53bef17a6484989a0e82b57e471fb/users/b8db9348ec5e4fda92118bbaf35cb21c/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:03.950454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a238bc-ecad-4038-b02a-3c748f4f9525 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:03.952140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a238bc-ecad-4038-b02a-3c748f4f9525 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:03.961372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a238bc-ecad-4038-b02a-3c748f4f9525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:04.012771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-97a238bc-ecad-4038-b02a-3c748f4f9525 None None] MFA Rules not processed for user `b8db9348ec5e4fda92118bbaf35cb21c`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:04.024350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-97a238bc-ecad-4038-b02a-3c748f4f9525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:04.166229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 947/1884] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:04.221039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:04.224088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:04.324571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:04.373754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a238bc-ecad-4038-b02a-3c748f4f9525 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:04.377929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-97a238bc-ecad-4038-b02a-3c748f4f9525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:04.379650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86d1e7-5128-4bd2-8e5a-1362b87f93c3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:04.390711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:04.392008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:04.392586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:04.393617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:04.435012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:04.438678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:04.459057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 938/1885] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:04.533049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a238bc-ecad-4038-b02a-3c748f4f9525 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:04.548637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:04.553380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:04.553380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:04.554327 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] PATH_INFO: `/v3/projects/acccaba63ff540b09caed4d6bcc3dc06` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:04.557396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=acccaba63ff540b09caed4d6bcc3dc06)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:04.563715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:04.818902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acccaba63ff540b09caed4d6bcc3dc06', 'request_id': 'req-8ef61a03-5c23-4742-b633-05838cd47e6c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:04.840685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acccaba63ff540b09caed4d6bcc3dc06', 'request_id': 'req-8ef61a03-5c23-4742-b633-05838cd47e6c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:04.922263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:04.928276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:04.935868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acccaba63ff540b09caed4d6bcc3dc06', 'request_id': 'req-8ef61a03-5c23-4742-b633-05838cd47e6c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:04.943512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025f01-2aa0-4037-bcc3-42fa9fae6484 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:04.944210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] The token cache is being invalidate because project acccaba63ff540b09caed4d6bcc3dc06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:04.944642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:04.945425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-8ef61a03-5c23-4742-b633-05838cd47e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:04.947136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 948/1886] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:04 2022] DELETE /identity/v3/projects/acccaba63ff540b09caed4d6bcc3dc06 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:04.955621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47971efd-3541-4e17-be7e-c32dc9beb9db None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:04.956027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47971efd-3541-4e17-be7e-c32dc9beb9db None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:04.956461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47971efd-3541-4e17-be7e-c32dc9beb9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:04.998958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-47971efd-3541-4e17-be7e-c32dc9beb9db None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:05.001630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-47971efd-3541-4e17-be7e-c32dc9beb9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:05.025361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:05.038941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:05.038941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:05.038941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:05.038941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:05.159688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:05.160729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 949/1887] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:05.162706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:05.164780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc3ecf-a09b-44e4-923f-6eb889665d52 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:05.166170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc3ecf-a09b-44e4-923f-6eb889665d52 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:05.172435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc3ecf-a09b-44e4-923f-6eb889665d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:05.177551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:05.195649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 939/1888] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:05.205210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-38bc3ecf-a09b-44e4-923f-6eb889665d52 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:05.206526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-38bc3ecf-a09b-44e4-923f-6eb889665d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:05.220741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 950/1889] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:05.223755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:05.245805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ec024-f7d9-4312-8ba2-2ee58ccb086a service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:05.256123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:05.257332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:05.258127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:05.258967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:05.260458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:05.261088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:05.264253 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:05.287079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 940/1890] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:05.292770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc3ecf-a09b-44e4-923f-6eb889665d52 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:05.305634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc3ecf-a09b-44e4-923f-6eb889665d52 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:05.312317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f10cc-1c45-44e8-8e3e-de742c5b1459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:05.313200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f10cc-1c45-44e8-8e3e-de742c5b1459 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:05.313528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f10cc-1c45-44e8-8e3e-de742c5b1459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:05.313811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f10cc-1c45-44e8-8e3e-de742c5b1459 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:05.314631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f10cc-1c45-44e8-8e3e-de742c5b1459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:05.317086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f10cc-1c45-44e8-8e3e-de742c5b1459 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:05.531737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 951/1891] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:05 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:05.536973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:05.550459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353d57d-2b30-4bfc-9f4a-e376ead073a8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:05.682482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993ac7c-7ff7-4fc8-9b94-c30d9f82299b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:05.683581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993ac7c-7ff7-4fc8-9b94-c30d9f82299b admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:05.683875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993ac7c-7ff7-4fc8-9b94-c30d9f82299b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:05.684214 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993ac7c-7ff7-4fc8-9b94-c30d9f82299b admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:05.684999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993ac7c-7ff7-4fc8-9b94-c30d9f82299b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:05.688409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993ac7c-7ff7-4fc8-9b94-c30d9f82299b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:06.035851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 941/1892] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:05 2022] POST /identity/v3/users => generated 291 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:06.040225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f10cc-1c45-44e8-8e3e-de742c5b1459 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:06.055107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f10cc-1c45-44e8-8e3e-de742c5b1459 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:06.059971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918744a9-86a4-4098-a69f-b9de0fca0dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:06.060964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918744a9-86a4-4098-a69f-b9de0fca0dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:06.061497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918744a9-86a4-4098-a69f-b9de0fca0dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:06.062673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918744a9-86a4-4098-a69f-b9de0fca0dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:06.063922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918744a9-86a4-4098-a69f-b9de0fca0dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:06.066035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918744a9-86a4-4098-a69f-b9de0fca0dc3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:06.088835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 952/1893] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:06.089175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993ac7c-7ff7-4fc8-9b94-c30d9f82299b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:06.113314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993ac7c-7ff7-4fc8-9b94-c30d9f82299b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:06.123674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd67bb-1093-4f7c-82f7-afe7910296db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:06.125643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd67bb-1093-4f7c-82f7-afe7910296db admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:06.126379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd67bb-1093-4f7c-82f7-afe7910296db admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:06.126942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd67bb-1093-4f7c-82f7-afe7910296db admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:06.129665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd67bb-1093-4f7c-82f7-afe7910296db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:06.133297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd67bb-1093-4f7c-82f7-afe7910296db admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:06.163601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 942/1894] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:06.167872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918744a9-86a4-4098-a69f-b9de0fca0dc3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:06.180682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918744a9-86a4-4098-a69f-b9de0fca0dc3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:06.187456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf873673-98fe-41f1-bb89-0aa5ac693e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:06.192426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf873673-98fe-41f1-bb89-0aa5ac693e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:06.192426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf873673-98fe-41f1-bb89-0aa5ac693e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:06.192426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf873673-98fe-41f1-bb89-0aa5ac693e9c admin admin] PATH_INFO: `/v3/projects/04dc9404552e482ebc83f9823005490d/users/04d6e5867ec94285bd588cebe97f73d3/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:06.219997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf873673-98fe-41f1-bb89-0aa5ac693e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=04dc9404552e482ebc83f9823005490d, user_id=04d6e5867ec94285bd588cebe97f73d3, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:06.225356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf873673-98fe-41f1-bb89-0aa5ac693e9c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:06.379899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 953/1895] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:06 2022] PUT /identity/v3/projects/04dc9404552e482ebc83f9823005490d/users/04d6e5867ec94285bd588cebe97f73d3/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:06.401996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce505e-8f42-44d9-82f4-f822969a4c38 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:06.402586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce505e-8f42-44d9-82f4-f822969a4c38 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:06.402871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce505e-8f42-44d9-82f4-f822969a4c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:06.481330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8cce505e-8f42-44d9-82f4-f822969a4c38 None None] MFA Rules not processed for user `04d6e5867ec94285bd588cebe97f73d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:06.482859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8cce505e-8f42-44d9-82f4-f822969a4c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:06.632394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 943/1896] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:06.639572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf873673-98fe-41f1-bb89-0aa5ac693e9c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:06.803814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf873673-98fe-41f1-bb89-0aa5ac693e9c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:06.811022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08cc5d-d978-4c3f-bb59-10d18e979554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:06.812485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08cc5d-d978-4c3f-bb59-10d18e979554 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:06.812963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08cc5d-d978-4c3f-bb59-10d18e979554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:06.813382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08cc5d-d978-4c3f-bb59-10d18e979554 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:06.814141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08cc5d-d978-4c3f-bb59-10d18e979554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:06.816871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08cc5d-d978-4c3f-bb59-10d18e979554 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:07.001772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce505e-8f42-44d9-82f4-f822969a4c38 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:07.011855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 954/1897] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:06 2022] POST /identity/v3/projects => generated 381 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:07.027559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce505e-8f42-44d9-82f4-f822969a4c38 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:07.163868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0247664-00d9-4eac-9e72-5658b0b00eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:07.164895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0247664-00d9-4eac-9e72-5658b0b00eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:07.164895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0247664-00d9-4eac-9e72-5658b0b00eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:07.165400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0247664-00d9-4eac-9e72-5658b0b00eb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:07.166280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0247664-00d9-4eac-9e72-5658b0b00eb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:07.168889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0247664-00d9-4eac-9e72-5658b0b00eb4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:07.313615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08cc5d-d978-4c3f-bb59-10d18e979554 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:07.320436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 944/1898] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:07 2022] POST /identity/v3/users => generated 297 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:07.336507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08cc5d-d978-4c3f-bb59-10d18e979554 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:07.348849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780509-25ad-43bb-a330-5c7e191e6b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:07.349692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780509-25ad-43bb-a330-5c7e191e6b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:07.351493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780509-25ad-43bb-a330-5c7e191e6b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:07.351493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a780509-25ad-43bb-a330-5c7e191e6b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:07.351493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a780509-25ad-43bb-a330-5c7e191e6b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:07.353174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a780509-25ad-43bb-a330-5c7e191e6b7d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:07.369571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 955/1899] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:07.376160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0247664-00d9-4eac-9e72-5658b0b00eb4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:07.395093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0247664-00d9-4eac-9e72-5658b0b00eb4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:07.404504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876a8b5-3a92-42c4-8987-1a35291bfe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:07.409122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876a8b5-3a92-42c4-8987-1a35291bfe2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:07.409122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876a8b5-3a92-42c4-8987-1a35291bfe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:07.409122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876a8b5-3a92-42c4-8987-1a35291bfe2a admin admin] PATH_INFO: `/v3/projects/5c837d17cfd94e0784a6fdb1cdad7513/users/30091cc6dcf04ea0b038ab52e96adcd1/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:07.410815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:07.440600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876a8b5-3a92-42c4-8987-1a35291bfe2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c837d17cfd94e0784a6fdb1cdad7513, user_id=30091cc6dcf04ea0b038ab52e96adcd1, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:07.446025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876a8b5-3a92-42c4-8987-1a35291bfe2a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:07.550103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 945/1900] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:07 2022] PUT /identity/v3/projects/5c837d17cfd94e0784a6fdb1cdad7513/users/30091cc6dcf04ea0b038ab52e96adcd1/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:07.554320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780509-25ad-43bb-a330-5c7e191e6b7d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:07.576011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a780509-25ad-43bb-a330-5c7e191e6b7d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:07.725788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ecec11-04d4-4155-b232-7d00936835b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:07.726978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecec11-04d4-4155-b232-7d00936835b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:07.727479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecec11-04d4-4155-b232-7d00936835b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:07.727979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ecec11-04d4-4155-b232-7d00936835b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:07.729313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecec11-04d4-4155-b232-7d00936835b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:07.732662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ecec11-04d4-4155-b232-7d00936835b0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:07.752815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 956/1901] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:07 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:07.770302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876a8b5-3a92-42c4-8987-1a35291bfe2a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:07.805700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876a8b5-3a92-42c4-8987-1a35291bfe2a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:07.825136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:07.825993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:07.826346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:07.826719 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] PATH_INFO: `/v3/projects/5c837d17cfd94e0784a6fdb1cdad7513/users/30091cc6dcf04ea0b038ab52e96adcd1/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:07.830276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c837d17cfd94e0784a6fdb1cdad7513, user_id=30091cc6dcf04ea0b038ab52e96adcd1, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:07.833484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:07.850710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 946/1902] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:07 2022] PUT /identity/v3/projects/5c837d17cfd94e0784a6fdb1cdad7513/users/30091cc6dcf04ea0b038ab52e96adcd1/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:07.865278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e2fb4-f82b-439e-b2de-db077819f170 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:07.865861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e2fb4-f82b-439e-b2de-db077819f170 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:07.866768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e2fb4-f82b-439e-b2de-db077819f170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:07.921848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-238e2fb4-f82b-439e-b2de-db077819f170 None None] MFA Rules not processed for user `30091cc6dcf04ea0b038ab52e96adcd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:07.924422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-238e2fb4-f82b-439e-b2de-db077819f170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:08.100235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 957/1903] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:08.270456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:08.273421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:08.423974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0f70e-4c91-43e8-8433-0f0f5878a5a4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:08.435645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68776536-e25a-416c-b844-c5b6c5b48dc6 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:08.436401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68776536-e25a-416c-b844-c5b6c5b48dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:08.437397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68776536-e25a-416c-b844-c5b6c5b48dc6 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:08.440238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 958/1904] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:39:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:39:08.443991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:08.445125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:08.445606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:08.446087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:08.450987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd189cc-1e8b-4d45-a94e-10cb010b4eef None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:08.451541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd189cc-1e8b-4d45-a94e-10cb010b4eef None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:08.453096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd189cc-1e8b-4d45-a94e-10cb010b4eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:08.471997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:08.473912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:08.492331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3dd189cc-1e8b-4d45-a94e-10cb010b4eef None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:08.508999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3dd189cc-1e8b-4d45-a94e-10cb010b4eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:08.533022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 947/1905] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:08.774068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 959/1906] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:39:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:08.795126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:08.797327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:08.851714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d782d7c-b15b-450c-a40e-41072e66f74f service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:08.881468 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:08.882788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:08.883685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:08.884237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:08.902896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:08.907098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:08.926848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 948/1907] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:08.962560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:09.502101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd189cc-1e8b-4d45-a94e-10cb010b4eef None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:09.522800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd189cc-1e8b-4d45-a94e-10cb010b4eef None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:09.529903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ca194-9863-4758-81d5-4e5b7e139467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:09.531305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ca194-9863-4758-81d5-4e5b7e139467 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:09.531842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ca194-9863-4758-81d5-4e5b7e139467 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:09.532167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ca194-9863-4758-81d5-4e5b7e139467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:09.534438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc7ca194-9863-4758-81d5-4e5b7e139467 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:09.610874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:09.613987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:09.649622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ca194-9863-4758-81d5-4e5b7e139467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:09.651099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ca194-9863-4758-81d5-4e5b7e139467 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:09.689509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 960/1908] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:09.788226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c73cc7-0388-48c8-8006-0a6e76cd3d67 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:09.793626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:09.794301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:09.794584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:09.794885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] PATH_INFO: `/v3/users/6972ec5f8d5a44a3bdaef9230d64c9cc` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:09.798471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6972ec5f8d5a44a3bdaef9230d64c9cc)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:09.804932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:10.007529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:10.164881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6972ec5f8d5a44a3bdaef9230d64c9cc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:10.165692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6972ec5f8d5a44a3bdaef9230d64c9cc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:10.176363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6972ec5f8d5a44a3bdaef9230d64c9cc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:10.177208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6972ec5f8d5a44a3bdaef9230d64c9cc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:10.298115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6972ec5f8d5a44a3bdaef9230d64c9cc'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:10.348640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 949/1909] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:09 2022] DELETE /identity/v3/users/6972ec5f8d5a44a3bdaef9230d64c9cc => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:10.354716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c35cb9-065d-4592-b168-dfe31a5483c3 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:10.355955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c35cb9-065d-4592-b168-dfe31a5483c3 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:10.356810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c35cb9-065d-4592-b168-dfe31a5483c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:10.397771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d9c35cb9-065d-4592-b168-dfe31a5483c3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:10.399336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d9c35cb9-065d-4592-b168-dfe31a5483c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:10.511676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 961/1910] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:10.526628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:10.529326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:10.742391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa9a5-ea68-4368-a8e2-0e01d681c130 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:10.757628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:10.758415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:10.769119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:10.769119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:10.772517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:10.775273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:10.793735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 950/1911] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:11.495054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c35cb9-065d-4592-b168-dfe31a5483c3 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:11.498198 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d9c35cb9-065d-4592-b168-dfe31a5483c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:11.528928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c35cb9-065d-4592-b168-dfe31a5483c3 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:11.546914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:11.548071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:11.548556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:11.549009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] PATH_INFO: `/v3/users/04d6e5867ec94285bd588cebe97f73d3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:11.551138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d6e5867ec94285bd588cebe97f73d3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:11.554025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:11.651568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:11.651568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:11.675344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa00586-5b65-4e5c-bd7e-68e54391a46f service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:11.685124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:11.686312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:11.687632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:11.687826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] PATH_INFO: `/v3/projects/d4e3aa40c9774cdb92f3aac19c24e2bf` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:11.689946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e3aa40c9774cdb92f3aac19c24e2bf)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:11.692964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:11.940117 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e3aa40c9774cdb92f3aac19c24e2bf', 'request_id': 'req-31e59355-9a18-4aaf-8741-19e5a8cb45e6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:11.955810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d6e5867ec94285bd588cebe97f73d3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:11.957003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d6e5867ec94285bd588cebe97f73d3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:11.961268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e3aa40c9774cdb92f3aac19c24e2bf', 'request_id': 'req-31e59355-9a18-4aaf-8741-19e5a8cb45e6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:11.974714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e3aa40c9774cdb92f3aac19c24e2bf', 'request_id': 'req-31e59355-9a18-4aaf-8741-19e5a8cb45e6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:11.979764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d6e5867ec94285bd588cebe97f73d3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.172772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] The token cache is being invalidate because project d4e3aa40c9774cdb92f3aac19c24e2bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:12.172772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.172772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.174821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 951/1912] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:11 2022] DELETE /identity/v3/projects/d4e3aa40c9774cdb92f3aac19c24e2bf => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:12.181353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:12.189783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:12.304954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e59355-9a18-4aaf-8741-19e5a8cb45e6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:12.312896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:12.313878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:12.314250 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:12.314708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] PATH_INFO: `/v3/users/06ba58daf0df47a2a86489a37b91aa1c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:12.316508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ba58daf0df47a2a86489a37b91aa1c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:12.318705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:12.380626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d6e5867ec94285bd588cebe97f73d3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.381581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ea39bc80-84ff-405b-8e18-a033e7133348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d6e5867ec94285bd588cebe97f73d3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.416102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 962/1913] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:11 2022] DELETE /identity/v3/users/04d6e5867ec94285bd588cebe97f73d3 => generated 0 bytes in 922 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:12.423949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c980bd-af6b-44e4-ae71-071c611f6ba9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:12.424474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c980bd-af6b-44e4-ae71-071c611f6ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:12.425257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c980bd-af6b-44e4-ae71-071c611f6ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:12.455056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e2c980bd-af6b-44e4-ae71-071c611f6ba9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:12.456446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e2c980bd-af6b-44e4-ae71-071c611f6ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:12.554553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 963/1914] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:12.571407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c980bd-af6b-44e4-ae71-071c611f6ba9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:12.573826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e2c980bd-af6b-44e4-ae71-071c611f6ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:12.644908 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ba58daf0df47a2a86489a37b91aa1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.646794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ba58daf0df47a2a86489a37b91aa1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.664133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ba58daf0df47a2a86489a37b91aa1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.666649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ba58daf0df47a2a86489a37b91aa1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.673719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c980bd-af6b-44e4-ae71-071c611f6ba9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:12.679560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:12.680472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:12.680869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:12.681465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:12.684124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:12.700865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:12.703720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:12.719560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 964/1915] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:12.849143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ba58daf0df47a2a86489a37b91aa1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:12.884385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 952/1916] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:12 2022] DELETE /identity/v3/users/06ba58daf0df47a2a86489a37b91aa1c => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:13.167753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:13.348097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:13.580919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:13.583679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:13.600251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:13.602665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:13.616505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb866f-3281-4dc1-9e96-b59028909726 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:13.750256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:13.751162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:13.751574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:13.751907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] PATH_INFO: `/v3/projects/96af120e1e0848b6877bac6cfc8c47b7` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:13.754022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96af120e1e0848b6877bac6cfc8c47b7)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:13.758827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:13.760088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e53a-7af3-4852-b7cc-6efc1b048ef7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:13.769851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:13.770912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:13.771392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:13.771923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] PATH_INFO: `/v3/projects/04dc9404552e482ebc83f9823005490d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:13.774256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04dc9404552e482ebc83f9823005490d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:13.778265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:13.950406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96af120e1e0848b6877bac6cfc8c47b7', 'request_id': 'req-ba76f6b0-0614-4b30-9a12-f67cfd413880'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:13.961648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96af120e1e0848b6877bac6cfc8c47b7', 'request_id': 'req-ba76f6b0-0614-4b30-9a12-f67cfd413880'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:13.983331 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04dc9404552e482ebc83f9823005490d', 'request_id': 'req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:14.002452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04dc9404552e482ebc83f9823005490d', 'request_id': 'req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:14.022746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04dc9404552e482ebc83f9823005490d', 'request_id': 'req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:14.205735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96af120e1e0848b6877bac6cfc8c47b7', 'request_id': 'req-ba76f6b0-0614-4b30-9a12-f67cfd413880'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:14.222562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] The token cache is being invalidate because project 96af120e1e0848b6877bac6cfc8c47b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:14.223056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:14.223970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ba76f6b0-0614-4b30-9a12-f67cfd413880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:14.226429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 965/1917] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:13 2022] DELETE /identity/v3/projects/96af120e1e0848b6877bac6cfc8c47b7 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:14.234144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b02b3-67e4-4972-97bd-dcdc2d34790c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:14.234616 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b02b3-67e4-4972-97bd-dcdc2d34790c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:14.235120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b02b3-67e4-4972-97bd-dcdc2d34790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:14.268793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a93b02b3-67e4-4972-97bd-dcdc2d34790c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:14.271038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a93b02b3-67e4-4972-97bd-dcdc2d34790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:14.341050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] The token cache is being invalidate because project 04dc9404552e482ebc83f9823005490d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:14.342078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:14.345803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:14.349508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 953/1918] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:13 2022] DELETE /identity/v3/projects/04dc9404552e482ebc83f9823005490d => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:14.361936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:14.365610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:14.419135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 966/1919] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:14.430020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f6c78-3e00-482d-91c5-97aa12b7f08e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:14.430351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f6c78-3e00-482d-91c5-97aa12b7f08e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:14.435845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f6c78-3e00-482d-91c5-97aa12b7f08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:14.470348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d50f6c78-3e00-482d-91c5-97aa12b7f08e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:14.472950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d50f6c78-3e00-482d-91c5-97aa12b7f08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:14.489374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 967/1920] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:14.495281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f6c78-3e00-482d-91c5-97aa12b7f08e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:14.521625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f6c78-3e00-482d-91c5-97aa12b7f08e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:14.535366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:14.536427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:14.536974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:14.537610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95ea50-24ec-490d-bfdf-adc5a77b3bb2 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:14.537777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:14.539464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:14.539987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:14.543274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:14.543704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:14.543999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:14.544339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:14.544624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] PATH_INFO: `/v3/users/30091cc6dcf04ea0b038ab52e96adcd1` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:14.546858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=30091cc6dcf04ea0b038ab52e96adcd1)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:14.549255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:14.572266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 968/1921] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:14.574978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:14.592310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b34ce1-cd72-42c2-88a0-3e851c0fe05e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:14.601483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0275825-598a-473b-a0ae-a275dc7b4853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:14.602462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0275825-598a-473b-a0ae-a275dc7b4853 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:14.602937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0275825-598a-473b-a0ae-a275dc7b4853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:14.603890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0275825-598a-473b-a0ae-a275dc7b4853 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:14.605066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0275825-598a-473b-a0ae-a275dc7b4853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:14.608621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0275825-598a-473b-a0ae-a275dc7b4853 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:14.922635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 969/1922] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:14 2022] POST /identity/v3/projects => generated 387 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:14.928491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0275825-598a-473b-a0ae-a275dc7b4853 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:15.027486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0275825-598a-473b-a0ae-a275dc7b4853 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:15.034623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:15.037243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:15.037713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:15.038049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:15.039735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:15.042660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:15.064821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30091cc6dcf04ea0b038ab52e96adcd1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:15.065913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30091cc6dcf04ea0b038ab52e96adcd1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:15.077944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30091cc6dcf04ea0b038ab52e96adcd1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:15.079123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30091cc6dcf04ea0b038ab52e96adcd1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:15.198496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30091cc6dcf04ea0b038ab52e96adcd1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:15.215904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 970/1923] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:14 2022] POST /identity/v3/users => generated 294 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:15.223425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:15.226872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:15.240582 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 954/1924] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:14 2022] DELETE /identity/v3/users/30091cc6dcf04ea0b038ab52e96adcd1 => generated 0 bytes in 880 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:15.246232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4821d7-07dc-459c-9461-8e8705712331 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:15.360401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3a87-da25-4915-bcc4-29b54fe8ff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:15.361490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3a87-da25-4915-bcc4-29b54fe8ff1e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:15.361829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3a87-da25-4915-bcc4-29b54fe8ff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:15.362180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3a87-da25-4915-bcc4-29b54fe8ff1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:15.363133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f3a87-da25-4915-bcc4-29b54fe8ff1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:15.365092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f3a87-da25-4915-bcc4-29b54fe8ff1e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:15.378201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 971/1925] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:15.386136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:15.406427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3ce24-bae9-4961-80b9-cf3c0cbcb0db admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:15.415168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f131e1-8802-4a40-94f3-0b50e486d1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:15.415811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f131e1-8802-4a40-94f3-0b50e486d1ab admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:15.416096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f131e1-8802-4a40-94f3-0b50e486d1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:15.416414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f131e1-8802-4a40-94f3-0b50e486d1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:15.417163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f131e1-8802-4a40-94f3-0b50e486d1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:15.418897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f131e1-8802-4a40-94f3-0b50e486d1ab admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:15.430913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 955/1926] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:15.435714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3a87-da25-4915-bcc4-29b54fe8ff1e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:15.453325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3a87-da25-4915-bcc4-29b54fe8ff1e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:15.462969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede883de-aa8b-498d-9ef2-01f86e62290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:15.464310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede883de-aa8b-498d-9ef2-01f86e62290d admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:15.464648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede883de-aa8b-498d-9ef2-01f86e62290d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:15.465127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede883de-aa8b-498d-9ef2-01f86e62290d admin admin] PATH_INFO: `/v3/projects/d69bcc81f8904858a1a47c8081e5ad3c/users/70fcb4a221844f8fb661cf64925dd618/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:15.510218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede883de-aa8b-498d-9ef2-01f86e62290d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69bcc81f8904858a1a47c8081e5ad3c, user_id=70fcb4a221844f8fb661cf64925dd618, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:15.521827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede883de-aa8b-498d-9ef2-01f86e62290d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:15.645626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 972/1927] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:15 2022] PUT /identity/v3/projects/d69bcc81f8904858a1a47c8081e5ad3c/users/70fcb4a221844f8fb661cf64925dd618/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:15.659481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada86d-df01-4a50-9482-5a1ac0f0dd39 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:15.659993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada86d-df01-4a50-9482-5a1ac0f0dd39 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:15.660476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ada86d-df01-4a50-9482-5a1ac0f0dd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:15.715124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-25ada86d-df01-4a50-9482-5a1ac0f0dd39 None None] MFA Rules not processed for user `70fcb4a221844f8fb661cf64925dd618`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:15.718156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-25ada86d-df01-4a50-9482-5a1ac0f0dd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:15.853452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 956/1928] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:15.855775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede883de-aa8b-498d-9ef2-01f86e62290d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:15.953877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:16.151624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ada86d-df01-4a50-9482-5a1ac0f0dd39 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:16.154010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-25ada86d-df01-4a50-9482-5a1ac0f0dd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:16.154155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede883de-aa8b-498d-9ef2-01f86e62290d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:16.165063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb35b-df07-4ef9-8df1-5d0d6f1b702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:16.166909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb35b-df07-4ef9-8df1-5d0d6f1b702a admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.167677 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb35b-df07-4ef9-8df1-5d0d6f1b702a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.168329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06fb35b-df07-4ef9-8df1-5d0d6f1b702a admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.170054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06fb35b-df07-4ef9-8df1-5d0d6f1b702a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:16.178527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06fb35b-df07-4ef9-8df1-5d0d6f1b702a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:16.193891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ada86d-df01-4a50-9482-5a1ac0f0dd39 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:16.208947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:16.209757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.213158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.213158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] PATH_INFO: `/v3/projects/5c837d17cfd94e0784a6fdb1cdad7513` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.216592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c837d17cfd94e0784a6fdb1cdad7513)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:16.223397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:16.363797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 973/1929] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:15 2022] POST /identity/v3/projects => generated 385 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:16.366392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb35b-df07-4ef9-8df1-5d0d6f1b702a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:16.469579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c837d17cfd94e0784a6fdb1cdad7513', 'request_id': 'req-bdc6b6a9-abd5-430f-8fb4-fd776188685d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:16.491223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c837d17cfd94e0784a6fdb1cdad7513', 'request_id': 'req-bdc6b6a9-abd5-430f-8fb4-fd776188685d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:16.501062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c837d17cfd94e0784a6fdb1cdad7513', 'request_id': 'req-bdc6b6a9-abd5-430f-8fb4-fd776188685d'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:16.570050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06fb35b-df07-4ef9-8df1-5d0d6f1b702a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:16.605199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] The token cache is being invalidate because project 5c837d17cfd94e0784a6fdb1cdad7513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:16.605671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:16.606931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-bdc6b6a9-abd5-430f-8fb4-fd776188685d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:16.610668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 957/1930] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:16 2022] DELETE /identity/v3/projects/5c837d17cfd94e0784a6fdb1cdad7513 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:16.618524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9c8c8-34b9-4f3c-9d5a-2f77decc2793 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.618892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9c8c8-34b9-4f3c-9d5a-2f77decc2793 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.619707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9c8c8-34b9-4f3c-9d5a-2f77decc2793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.662895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cba9c8c8-34b9-4f3c-9d5a-2f77decc2793 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:16.664994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cba9c8c8-34b9-4f3c-9d5a-2f77decc2793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:16.679765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:16.681000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.681538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.682163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.683590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:16.687745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:16.777351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 958/1931] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:16.786898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835272-e0e3-4f21-acf8-6a6fe625b389 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.787184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835272-e0e3-4f21-acf8-6a6fe625b389 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.787459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd835272-e0e3-4f21-acf8-6a6fe625b389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.811966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-dd835272-e0e3-4f21-acf8-6a6fe625b389 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:16.813470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dd835272-e0e3-4f21-acf8-6a6fe625b389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:16.822733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 959/1932] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:16.836916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd835272-e0e3-4f21-acf8-6a6fe625b389 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:16.838531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd835272-e0e3-4f21-acf8-6a6fe625b389 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:16.844511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:16.845378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.845661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.845937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.846614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:16.846921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:16.848947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:16.872720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 960/1933] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:39:16.877833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:16.901609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 974/1934] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:16 2022] POST /identity/v3/users => generated 299 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:16.901867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13458c28-e1d1-431c-bb0a-b55fcf1d2bf1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:16.903111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:16.904858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0101e-6866-42f9-9f26-4473837c91ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:16.906294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:16.906500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0101e-6866-42f9-9f26-4473837c91ca admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.907018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0101e-6866-42f9-9f26-4473837c91ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.907448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0101e-6866-42f9-9f26-4473837c91ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.908778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0101e-6866-42f9-9f26-4473837c91ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:16.912874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0101e-6866-42f9-9f26-4473837c91ca admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:16.922852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135972b-c63d-4402-8b64-283d4f75b5ff admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:16.930785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6a1f4-4ee0-4621-9d6f-ddad1b359e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:16.931622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a1f4-4ee0-4621-9d6f-ddad1b359e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.932067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a1f4-4ee0-4621-9d6f-ddad1b359e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.932387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6a1f4-4ee0-4621-9d6f-ddad1b359e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.933136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6a1f4-4ee0-4621-9d6f-ddad1b359e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:16.935704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6a1f4-4ee0-4621-9d6f-ddad1b359e03 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:16.949185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 975/1935] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:16.954255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6a1f4-4ee0-4621-9d6f-ddad1b359e03 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:16.965504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6a1f4-4ee0-4621-9d6f-ddad1b359e03 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:16.971333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2be6c-cd91-43c0-a910-2e9004b94e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:16.972026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2be6c-cd91-43c0-a910-2e9004b94e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:16.972355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2be6c-cd91-43c0-a910-2e9004b94e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:16.972636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2be6c-cd91-43c0-a910-2e9004b94e88 admin admin] PATH_INFO: `/v3/projects/0ee95110d89547d9a06ab2464a222587/users/6732f03930794cdfbce762394d3e8956/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:16.992856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2be6c-cd91-43c0-a910-2e9004b94e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee95110d89547d9a06ab2464a222587, user_id=6732f03930794cdfbce762394d3e8956, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:16.995535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2be6c-cd91-43c0-a910-2e9004b94e88 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:17.034348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:17.047232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 961/1936] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:16 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:17.049369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0101e-6866-42f9-9f26-4473837c91ca admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:17.105909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 976/1937] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:16 2022] PUT /identity/v3/projects/0ee95110d89547d9a06ab2464a222587/users/6732f03930794cdfbce762394d3e8956/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:17.110669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2be6c-cd91-43c0-a910-2e9004b94e88 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:17.125440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0101e-6866-42f9-9f26-4473837c91ca admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:17.239221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2be6c-cd91-43c0-a910-2e9004b94e88 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:17.240882 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2e928-7918-491e-98ae-febcc1329fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:17.242395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2e928-7918-491e-98ae-febcc1329fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:17.247090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2e928-7918-491e-98ae-febcc1329fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:17.247090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2e928-7918-491e-98ae-febcc1329fe1 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:17.247090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2e928-7918-491e-98ae-febcc1329fe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:17.248764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2e928-7918-491e-98ae-febcc1329fe1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:17.249488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f1a56-e311-4bcd-b5e5-2ecff43e7466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:17.250579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1a56-e311-4bcd-b5e5-2ecff43e7466 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:17.251005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1a56-e311-4bcd-b5e5-2ecff43e7466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:17.251462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1a56-e311-4bcd-b5e5-2ecff43e7466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:17.252974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f1a56-e311-4bcd-b5e5-2ecff43e7466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:17.256093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f1a56-e311-4bcd-b5e5-2ecff43e7466 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:17.276334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 977/1938] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:17.282315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f1a56-e311-4bcd-b5e5-2ecff43e7466 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:17.295063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f1a56-e311-4bcd-b5e5-2ecff43e7466 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:17.303244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf262d-cfe2-4f4f-baaa-c11c1dc5d3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:17.304003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf262d-cfe2-4f4f-baaa-c11c1dc5d3d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:17.304319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf262d-cfe2-4f4f-baaa-c11c1dc5d3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:17.304605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf262d-cfe2-4f4f-baaa-c11c1dc5d3d3 admin admin] PATH_INFO: `/v3/projects/0ee95110d89547d9a06ab2464a222587/users/6732f03930794cdfbce762394d3e8956/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:17.306745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf262d-cfe2-4f4f-baaa-c11c1dc5d3d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee95110d89547d9a06ab2464a222587, user_id=6732f03930794cdfbce762394d3e8956, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:17.320823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf262d-cfe2-4f4f-baaa-c11c1dc5d3d3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:17.322772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 978/1939] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:17 2022] PUT /identity/v3/projects/0ee95110d89547d9a06ab2464a222587/users/6732f03930794cdfbce762394d3e8956/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:17.337735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fab001-4703-4979-81fb-127e8adc7fcd None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:17.338261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fab001-4703-4979-81fb-127e8adc7fcd None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:17.338633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fab001-4703-4979-81fb-127e8adc7fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:17.372383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-57fab001-4703-4979-81fb-127e8adc7fcd None None] MFA Rules not processed for user `6732f03930794cdfbce762394d3e8956`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:17.373744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-57fab001-4703-4979-81fb-127e8adc7fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:17.454549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 962/1940] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:17 2022] POST /identity/v3/users => generated 289 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:17.460612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2e928-7918-491e-98ae-febcc1329fe1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:17.485595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2e928-7918-491e-98ae-febcc1329fe1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:17.493000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 979/1941] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:17.634066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3429-68fc-4246-a735-d922946db1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:17.634746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3429-68fc-4246-a735-d922946db1dd admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:17.635040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3429-68fc-4246-a735-d922946db1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:17.635321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3429-68fc-4246-a735-d922946db1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:17.640846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c3429-68fc-4246-a735-d922946db1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:17.642796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c3429-68fc-4246-a735-d922946db1dd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:17.659299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 963/1942] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:17.665312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fab001-4703-4979-81fb-127e8adc7fcd None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:17.683110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fab001-4703-4979-81fb-127e8adc7fcd None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:17.697005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:17.698967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:17.700800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:17.701294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:17.703761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:17.710096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:17.726357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 980/1943] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:17.732457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3429-68fc-4246-a735-d922946db1dd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:17.745800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3429-68fc-4246-a735-d922946db1dd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:17.755503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec87f-e253-4a68-80e1-500ee8cc4042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:17.756120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec87f-e253-4a68-80e1-500ee8cc4042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:17.757127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec87f-e253-4a68-80e1-500ee8cc4042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:17.757127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec87f-e253-4a68-80e1-500ee8cc4042 admin admin] PATH_INFO: `/v3/projects/4e435478690c4063a3446055972ca54c/users/81759b101bc44b8d99638d5735e9818f/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:17.773824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:17.780228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cec87f-e253-4a68-80e1-500ee8cc4042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e435478690c4063a3446055972ca54c, user_id=81759b101bc44b8d99638d5735e9818f, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:17.783881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cec87f-e253-4a68-80e1-500ee8cc4042 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:17.927506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:17.932792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:17.940828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 964/1944] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:17 2022] PUT /identity/v3/projects/4e435478690c4063a3446055972ca54c/users/81759b101bc44b8d99638d5735e9818f/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:17.958721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443aef-ed51-480b-9e6d-7eaa120a6966 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:17.959642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443aef-ed51-480b-9e6d-7eaa120a6966 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:17.960346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443aef-ed51-480b-9e6d-7eaa120a6966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:18.062896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb9d7b-c643-4160-a67d-fa9a328ce3a1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:18.065826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-55443aef-ed51-480b-9e6d-7eaa120a6966 None None] MFA Rules not processed for user `81759b101bc44b8d99638d5735e9818f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:18.068212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-55443aef-ed51-480b-9e6d-7eaa120a6966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:18.175976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:18.176653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:18.176937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:18.177326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:18.236466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 965/1945] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:18.280088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:18.281761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:18.293424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 981/1946] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:18.406153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55443aef-ed51-480b-9e6d-7eaa120a6966 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:18.434133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55443aef-ed51-480b-9e6d-7eaa120a6966 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:18.441766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74ef22-9e36-409a-8a93-2678a742be2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:18.442859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74ef22-9e36-409a-8a93-2678a742be2b service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:18.443271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74ef22-9e36-409a-8a93-2678a742be2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:18.443758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74ef22-9e36-409a-8a93-2678a742be2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:18.460342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74ef22-9e36-409a-8a93-2678a742be2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:18.462325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74ef22-9e36-409a-8a93-2678a742be2b service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:18.475133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 966/1947] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:18.529952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:19.034789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:19.261595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:19.264931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:19.465481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74ef22-9e36-409a-8a93-2678a742be2b service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:19.529499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0dc039-6d2e-42f5-a278-cc20e8e0e899 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:19.565519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:19.566707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:19.567224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:19.568121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] PATH_INFO: `/v3/users/cc643e53d16c4ecfa04e0a3bf2e0ffa8` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:19.576884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc643e53d16c4ecfa04e0a3bf2e0ffa8)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:19.580204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:19.697871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74ef22-9e36-409a-8a93-2678a742be2b service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:19.723980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:19.726501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:19.727031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:19.727516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] PATH_INFO: `/v3/users/70fcb4a221844f8fb661cf64925dd618` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:19.734538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=70fcb4a221844f8fb661cf64925dd618)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:19.738227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:20.046781 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:20.070008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc643e53d16c4ecfa04e0a3bf2e0ffa8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.071098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc643e53d16c4ecfa04e0a3bf2e0ffa8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.086223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc643e53d16c4ecfa04e0a3bf2e0ffa8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.176825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70fcb4a221844f8fb661cf64925dd618'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.177941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70fcb4a221844f8fb661cf64925dd618'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.178171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc643e53d16c4ecfa04e0a3bf2e0ffa8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.182454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a6e68f3f-8879-4756-80ee-a54ef4475266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc643e53d16c4ecfa04e0a3bf2e0ffa8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.191070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70fcb4a221844f8fb661cf64925dd618'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.192418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70fcb4a221844f8fb661cf64925dd618'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.235124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 982/1948] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:19 2022] DELETE /identity/v3/users/cc643e53d16c4ecfa04e0a3bf2e0ffa8 => generated 0 bytes in 975 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:20.240248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76844626-fa1c-4c58-bb02-0e1301b07936 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:20.240724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76844626-fa1c-4c58-bb02-0e1301b07936 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:20.241114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76844626-fa1c-4c58-bb02-0e1301b07936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:20.272675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-76844626-fa1c-4c58-bb02-0e1301b07936 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:20.275897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-76844626-fa1c-4c58-bb02-0e1301b07936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:20.348149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-eb0fa1e4-5571-49bc-aede-1a834d7468bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70fcb4a221844f8fb661cf64925dd618'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:20.414518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 967/1949] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:19 2022] DELETE /identity/v3/users/70fcb4a221844f8fb661cf64925dd618 => generated 0 bytes in 950 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:20.420158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44294d14-6a45-400f-893a-44bfb701f88f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:20.422555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44294d14-6a45-400f-893a-44bfb701f88f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:20.423388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44294d14-6a45-400f-893a-44bfb701f88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:20.429830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 983/1950] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:20.458937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-44294d14-6a45-400f-893a-44bfb701f88f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:20.461008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-44294d14-6a45-400f-893a-44bfb701f88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:20.491279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76844626-fa1c-4c58-bb02-0e1301b07936 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:20.493524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-76844626-fa1c-4c58-bb02-0e1301b07936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:20.631358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 968/1951] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:20.651462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44294d14-6a45-400f-893a-44bfb701f88f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:20.662459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76844626-fa1c-4c58-bb02-0e1301b07936 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:20.804819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44294d14-6a45-400f-893a-44bfb701f88f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:20.810997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:20.813799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:20.815445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:20.815607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:20.824392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:20.824392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:20.824392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:20.824862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:20.847448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:20.848573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:20.848958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:20.851854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:20.870976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 984/1952] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:20.941042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 969/1953] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:21.243971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:21.582616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:22.020404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:22.214216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:22.216115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:22.246314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92a53e-430d-4d0c-bbd1-26da34a5b988 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:22.253774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:22.255388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:22.255682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:22.255962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] PATH_INFO: `/v3/users/964935f2a70f49fbab0a1644fae8c4a1` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:22.257882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] RBAC: Authorizing `identity:delete_user(user_id=964935f2a70f49fbab0a1644fae8c4a1)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:22.260103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:22.488222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:22.492988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:22.518388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee6064-f125-4100-9676-e4e1d35dcb08 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:22.526843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:22.527921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:22.531129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:22.531985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] PATH_INFO: `/v3/projects/d69bcc81f8904858a1a47c8081e5ad3c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:22.535048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69bcc81f8904858a1a47c8081e5ad3c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:22.538771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:22.574616 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964935f2a70f49fbab0a1644fae8c4a1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.575458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964935f2a70f49fbab0a1644fae8c4a1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.589517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964935f2a70f49fbab0a1644fae8c4a1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.690388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964935f2a70f49fbab0a1644fae8c4a1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.691130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964935f2a70f49fbab0a1644fae8c4a1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.729924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 985/1954] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:22 2022] DELETE /identity/v3/users/964935f2a70f49fbab0a1644fae8c4a1 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:22.730073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:22.730895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69bcc81f8904858a1a47c8081e5ad3c', 'request_id': 'req-75e277c4-d345-46b2-9a95-279af05127b8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.734849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:22.744535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69bcc81f8904858a1a47c8081e5ad3c', 'request_id': 'req-75e277c4-d345-46b2-9a95-279af05127b8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.758291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69bcc81f8904858a1a47c8081e5ad3c', 'request_id': 'req-75e277c4-d345-46b2-9a95-279af05127b8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.853622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9775309-d65f-4c38-999f-4d03406fcfad admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:22.860477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:22.864535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:22.866806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:22.866806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] PATH_INFO: `/v3/projects/f74f68f00fdc4a9e934fe1a4dcda6bc4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:22.866806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74f68f00fdc4a9e934fe1a4dcda6bc4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:22.868711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:22.950736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] The token cache is being invalidate because project d69bcc81f8904858a1a47c8081e5ad3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:22.951212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.952918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-75e277c4-d345-46b2-9a95-279af05127b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:22.955325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 970/1955] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:22 2022] DELETE /identity/v3/projects/d69bcc81f8904858a1a47c8081e5ad3c => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:22.957815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28e4a1-a991-4ea1-b685-a54e367ae0b9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:22.958563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28e4a1-a991-4ea1-b685-a54e367ae0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:22.958958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28e4a1-a991-4ea1-b685-a54e367ae0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:22.974353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:22.995919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4f28e4a1-a991-4ea1-b685-a54e367ae0b9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:22.998019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4f28e4a1-a991-4ea1-b685-a54e367ae0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:23.034354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 971/1956] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:23.036342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28e4a1-a991-4ea1-b685-a54e367ae0b9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:23.040531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4f28e4a1-a991-4ea1-b685-a54e367ae0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:23.051656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74f68f00fdc4a9e934fe1a4dcda6bc4', 'request_id': 'req-ce23f302-cde0-47af-9b58-c7c47a411bc1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.063438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74f68f00fdc4a9e934fe1a4dcda6bc4', 'request_id': 'req-ce23f302-cde0-47af-9b58-c7c47a411bc1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.065201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28e4a1-a991-4ea1-b685-a54e367ae0b9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:23.165688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:23.166584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:23.166973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:23.167352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] PATH_INFO: `/v3/users/6732f03930794cdfbce762394d3e8956` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:23.168813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6732f03930794cdfbce762394d3e8956)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:23.171191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:23.182187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74f68f00fdc4a9e934fe1a4dcda6bc4', 'request_id': 'req-ce23f302-cde0-47af-9b58-c7c47a411bc1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.198470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] The token cache is being invalidate because project f74f68f00fdc4a9e934fe1a4dcda6bc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:23.211409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.211409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.211409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 986/1957] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:22 2022] DELETE /identity/v3/projects/f74f68f00fdc4a9e934fe1a4dcda6bc4 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:23.211409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:23.211409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:23.312075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23f302-cde0-47af-9b58-c7c47a411bc1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:23.316239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:23.316899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:23.317177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:23.317545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:23.320018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:23.336542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:23.339359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:23.355235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 987/1958] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:23.364451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:23.370862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:23.411845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8df9c8-9f23-4839-a1c1-10e761352dc8 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:23.447762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:23.447762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:23.447762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:23.447762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] PATH_INFO: `/v3/users/d14635311a3642a7ac9535350c4da28a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:23.448505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14635311a3642a7ac9535350c4da28a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:23.448505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:23.613939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6732f03930794cdfbce762394d3e8956'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.614674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6732f03930794cdfbce762394d3e8956'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.622979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6732f03930794cdfbce762394d3e8956'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.624285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6732f03930794cdfbce762394d3e8956'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.750001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6732f03930794cdfbce762394d3e8956'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.769972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14635311a3642a7ac9535350c4da28a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.784255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14635311a3642a7ac9535350c4da28a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.797138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14635311a3642a7ac9535350c4da28a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.816737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 972/1959] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:23 2022] DELETE /identity/v3/users/6732f03930794cdfbce762394d3e8956 => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:23.911312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14635311a3642a7ac9535350c4da28a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.911647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14635311a3642a7ac9535350c4da28a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:23.966986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 988/1960] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:23 2022] DELETE /identity/v3/users/d14635311a3642a7ac9535350c4da28a => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:24.039181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:24.255129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:24.257653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:24.414763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267087-67e4-4a1e-995f-03cd300dd9a0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:24.551625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:24.555620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:24.555808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:24.556258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] PATH_INFO: `/v3/projects/f61d348e10bc46eba7ff80f590c14002` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:24.558844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61d348e10bc46eba7ff80f590c14002)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:24.561257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:24.758481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61d348e10bc46eba7ff80f590c14002', 'request_id': 'req-c670906a-493a-46d3-9f81-e3fb69d77771'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:24.766429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61d348e10bc46eba7ff80f590c14002', 'request_id': 'req-c670906a-493a-46d3-9f81-e3fb69d77771'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:24.779427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:24.784978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:24.788533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61d348e10bc46eba7ff80f590c14002', 'request_id': 'req-c670906a-493a-46d3-9f81-e3fb69d77771'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:24.806492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54922560-8bd3-4f66-821d-d7f6aa0169c3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:24.851112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:24.908486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:24.908486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:24.908486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:24.908486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] PATH_INFO: `/v3/projects/0ee95110d89547d9a06ab2464a222587` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:24.910400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee95110d89547d9a06ab2464a222587)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:24.916434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:24.927821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] The token cache is being invalidate because project f61d348e10bc46eba7ff80f590c14002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:24.928307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:24.929348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:24.931976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 973/1961] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:24 2022] DELETE /identity/v3/projects/f61d348e10bc46eba7ff80f590c14002 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:24.934177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:24.937131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:24.963714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670906a-493a-46d3-9f81-e3fb69d77771 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:24.971014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:24.971827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:24.972309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:24.972666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] PATH_INFO: `/v3/projects/f86ba1a9d2c942699c78478aa84f946e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:24.974379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86ba1a9d2c942699c78478aa84f946e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:24.976804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:25.059892 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee95110d89547d9a06ab2464a222587', 'request_id': 'req-9b9aa21b-0181-4fbb-9273-145d432492ad'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.067364 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee95110d89547d9a06ab2464a222587', 'request_id': 'req-9b9aa21b-0181-4fbb-9273-145d432492ad'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.160398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86ba1a9d2c942699c78478aa84f946e', 'request_id': 'req-dc939e09-c574-45ad-8222-0e4f91f889ef'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.172843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86ba1a9d2c942699c78478aa84f946e', 'request_id': 'req-dc939e09-c574-45ad-8222-0e4f91f889ef'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.185625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86ba1a9d2c942699c78478aa84f946e', 'request_id': 'req-dc939e09-c574-45ad-8222-0e4f91f889ef'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.188525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee95110d89547d9a06ab2464a222587', 'request_id': 'req-9b9aa21b-0181-4fbb-9273-145d432492ad'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.214388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] The token cache is being invalidate because project 0ee95110d89547d9a06ab2464a222587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:25.215268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.218283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.223461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 989/1962] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:24 2022] DELETE /identity/v3/projects/0ee95110d89547d9a06ab2464a222587 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:25.226202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:25.228981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:25.328933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] The token cache is being invalidate because project f86ba1a9d2c942699c78478aa84f946e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:25.331140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.331140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dc939e09-c574-45ad-8222-0e4f91f889ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.333365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 974/1963] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:24 2022] DELETE /identity/v3/projects/f86ba1a9d2c942699c78478aa84f946e => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:25.339088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2f135-499f-42e0-91c7-c7b27b4328d3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:25.339739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2f135-499f-42e0-91c7-c7b27b4328d3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:25.340363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2f135-499f-42e0-91c7-c7b27b4328d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:25.346346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9aa21b-0181-4fbb-9273-145d432492ad admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:25.366684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c1f2f135-499f-42e0-91c7-c7b27b4328d3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:25.368856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c1f2f135-499f-42e0-91c7-c7b27b4328d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:25.439722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:25.441090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:25.441543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:25.442030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] PATH_INFO: `/v3/users/b8db9348ec5e4fda92118bbaf35cb21c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:25.444402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8db9348ec5e4fda92118bbaf35cb21c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:25.451938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:25.469995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 975/1964] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:25.469995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2f135-499f-42e0-91c7-c7b27b4328d3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:25.472458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c1f2f135-499f-42e0-91c7-c7b27b4328d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:25.497328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2f135-499f-42e0-91c7-c7b27b4328d3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:25.507784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:25.514767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:25.514767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:25.514767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] PATH_INFO: `/v3/users/a9c707d1ec1a41c0b6a4c0466799efd0` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:25.514767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c707d1ec1a41c0b6a4c0466799efd0)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:25.515971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:25.734680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8db9348ec5e4fda92118bbaf35cb21c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.735417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8db9348ec5e4fda92118bbaf35cb21c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.744342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8db9348ec5e4fda92118bbaf35cb21c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.818189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c707d1ec1a41c0b6a4c0466799efd0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.818938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c707d1ec1a41c0b6a4c0466799efd0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.827296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c707d1ec1a41c0b6a4c0466799efd0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.828046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c707d1ec1a41c0b6a4c0466799efd0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.839524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8db9348ec5e4fda92118bbaf35cb21c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.840634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9b88989a-ef41-4cfe-bcc6-a3e2a33236e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8db9348ec5e4fda92118bbaf35cb21c'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.860642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 990/1965] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:25 2022] DELETE /identity/v3/users/b8db9348ec5e4fda92118bbaf35cb21c => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:25.861958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235aa5b-a669-44b8-9f93-2d85b84ff5cb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:25.862253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235aa5b-a669-44b8-9f93-2d85b84ff5cb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:25.862534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235aa5b-a669-44b8-9f93-2d85b84ff5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:25.884717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4235aa5b-a669-44b8-9f93-2d85b84ff5cb None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:25.886992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4235aa5b-a669-44b8-9f93-2d85b84ff5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:25.910076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:25.942291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c707d1ec1a41c0b6a4c0466799efd0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:25.997077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 991/1966] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:25.998123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235aa5b-a669-44b8-9f93-2d85b84ff5cb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:26.003834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 976/1967] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:25 2022] DELETE /identity/v3/users/a9c707d1ec1a41c0b6a4c0466799efd0 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:26.017675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235aa5b-a669-44b8-9f93-2d85b84ff5cb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:26.136202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:26.136681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.137872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.137872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.138220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:26.138498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:26.143136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:26.157592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 992/1968] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:26.162597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:26.184729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a97943-16ce-42c3-8ef8-4cd5dca72138 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:26.200483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:26.202163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.203313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.203313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.204496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:26.214922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:26.344638 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:26.364132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 977/1969] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:26 2022] POST /identity/v3/projects => generated 407 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:26.367461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:26.378991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2e54-d0e5-4448-8f6e-10bc603a50a9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:26.445845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dae0d-9f79-467a-82d8-ea808f1c8279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:26.446564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dae0d-9f79-467a-82d8-ea808f1c8279 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.446927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dae0d-9f79-467a-82d8-ea808f1c8279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.447206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6dae0d-9f79-467a-82d8-ea808f1c8279 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.447922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dae0d-9f79-467a-82d8-ea808f1c8279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:26.449814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6dae0d-9f79-467a-82d8-ea808f1c8279 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:26.456567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:26.458337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:26.482036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014537e6-37cf-439a-9cd4-615174b33354 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:26.489701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:26.490050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.490320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.490556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] PATH_INFO: `/v3/projects/81a53bef17a6484989a0e82b57e471fb` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.492084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a53bef17a6484989a0e82b57e471fb)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:26.494628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:26.546036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 993/1970] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:26 2022] POST /identity/v3/users => generated 304 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:26.551495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dae0d-9f79-467a-82d8-ea808f1c8279 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:26.563858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6dae0d-9f79-467a-82d8-ea808f1c8279 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:26.569058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47cf19-c054-40fe-ba74-ea2f4a137048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:26.569885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47cf19-c054-40fe-ba74-ea2f4a137048 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.570130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47cf19-c054-40fe-ba74-ea2f4a137048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.570255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47cf19-c054-40fe-ba74-ea2f4a137048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.571048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47cf19-c054-40fe-ba74-ea2f4a137048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:26.572775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47cf19-c054-40fe-ba74-ea2f4a137048 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:26.585324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 994/1971] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:26.589500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47cf19-c054-40fe-ba74-ea2f4a137048 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:26.601020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47cf19-c054-40fe-ba74-ea2f4a137048 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:26.604680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a53bef17a6484989a0e82b57e471fb', 'request_id': 'req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:26.613164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a53bef17a6484989a0e82b57e471fb', 'request_id': 'req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:26.638568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a53bef17a6484989a0e82b57e471fb', 'request_id': 'req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:26.701745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:26.706507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:26.707219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.707713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.708119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.708928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:26.710653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:26.722789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 995/1972] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:26.723656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:26.725448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:26.737609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] The token cache is being invalidate because project 81a53bef17a6484989a0e82b57e471fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:26.738356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:26.739199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:26.741413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 978/1973] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:26 2022] DELETE /identity/v3/projects/81a53bef17a6484989a0e82b57e471fb => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:26.742496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:26.744373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:26.747951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca21e60-1b79-4393-9736-d67af469f11c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:26.756694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:26.757780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.758291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.758884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] PATH_INFO: `/v3/projects/d283bc60ff7647a2a5aed68fc26028c1` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.760918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d283bc60ff7647a2a5aed68fc26028c1)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:26.761531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196df4d-8aaa-4a13-94a4-aaf7cb2f7558 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:26.764951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:26.767294 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836ca91-4dbe-49b7-a8e1-3c0b0dc8ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:26.768535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836ca91-4dbe-49b7-a8e1-3c0b0dc8ec39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.772923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836ca91-4dbe-49b7-a8e1-3c0b0dc8ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.772923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836ca91-4dbe-49b7-a8e1-3c0b0dc8ec39 admin admin] PATH_INFO: `/v3/projects/cae55ca3c2a943259ce886c22848d639/users/c5fdcaed01a74ef0ac9279d6007a3283/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.786182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836ca91-4dbe-49b7-a8e1-3c0b0dc8ec39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae55ca3c2a943259ce886c22848d639, user_id=c5fdcaed01a74ef0ac9279d6007a3283, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:26.789226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836ca91-4dbe-49b7-a8e1-3c0b0dc8ec39 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:26.900860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 979/1974] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:26 2022] PUT /identity/v3/projects/cae55ca3c2a943259ce886c22848d639/users/c5fdcaed01a74ef0ac9279d6007a3283/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:26.902984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63e149-3ab7-466c-8d14-a06430cdfcab None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:26.904567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63e149-3ab7-466c-8d14-a06430cdfcab None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:26.905889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63e149-3ab7-466c-8d14-a06430cdfcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:26.926317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd283bc60ff7647a2a5aed68fc26028c1', 'request_id': 'req-eb081a7f-565a-4263-b7ac-0df448739d05'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:26.927837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ba63e149-3ab7-466c-8d14-a06430cdfcab None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:26.929176 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ba63e149-3ab7-466c-8d14-a06430cdfcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:26.935014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd283bc60ff7647a2a5aed68fc26028c1', 'request_id': 'req-eb081a7f-565a-4263-b7ac-0df448739d05'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:27.012581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 980/1975] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:27.014500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04980a79-768f-422b-b3bc-124ecc6ce646 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.014975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04980a79-768f-422b-b3bc-124ecc6ce646 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.015372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04980a79-768f-422b-b3bc-124ecc6ce646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.016377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd283bc60ff7647a2a5aed68fc26028c1', 'request_id': 'req-eb081a7f-565a-4263-b7ac-0df448739d05'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:27.032754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] The token cache is being invalidate because project d283bc60ff7647a2a5aed68fc26028c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:27.033562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:27.034643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-eb081a7f-565a-4263-b7ac-0df448739d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:27.036546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 996/1976] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:26 2022] DELETE /identity/v3/projects/d283bc60ff7647a2a5aed68fc26028c1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:27.038812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2f16-67e6-436e-b56c-dfde164bafd1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.039752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2f16-67e6-436e-b56c-dfde164bafd1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.039863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e2f16-67e6-436e-b56c-dfde164bafd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.063925 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-04980a79-768f-422b-b3bc-124ecc6ce646 None None] MFA Rules not processed for user `c5fdcaed01a74ef0ac9279d6007a3283`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:27.066015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-04980a79-768f-422b-b3bc-124ecc6ce646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:27.070718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-354e2f16-67e6-436e-b56c-dfde164bafd1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:27.072284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-354e2f16-67e6-436e-b56c-dfde164bafd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:27.089354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 997/1977] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:27.089354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe1ef5-ea1e-4aa7-a1c8-c90070a2a9df None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.089354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe1ef5-ea1e-4aa7-a1c8-c90070a2a9df None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.089354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe1ef5-ea1e-4aa7-a1c8-c90070a2a9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.112916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-07fe1ef5-ea1e-4aa7-a1c8-c90070a2a9df None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:27.126525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-07fe1ef5-ea1e-4aa7-a1c8-c90070a2a9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:27.141285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 998/1978] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:27.152762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe1ef5-ea1e-4aa7-a1c8-c90070a2a9df None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.163575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe1ef5-ea1e-4aa7-a1c8-c90070a2a9df None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.171721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.172848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.173346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.173843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.175317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:27.176209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.179459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.197899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 981/1979] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:27.199348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95132606-6e1e-4843-bf8c-5b71ec44adc9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.199753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95132606-6e1e-4843-bf8c-5b71ec44adc9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.200034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95132606-6e1e-4843-bf8c-5b71ec44adc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.204232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 999/1980] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:27.205970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.208941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:27.222412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-95132606-6e1e-4843-bf8c-5b71ec44adc9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:27.223696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-95132606-6e1e-4843-bf8c-5b71ec44adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:27.232358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 982/1981] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:27.232784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4563c26-986e-4251-8778-edf10c7ec842 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.233388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95132606-6e1e-4843-bf8c-5b71ec44adc9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.241204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06046d9-9582-453f-9b62-74c4d12cd402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.242247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06046d9-9582-453f-9b62-74c4d12cd402 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.242755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06046d9-9582-453f-9b62-74c4d12cd402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.243129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06046d9-9582-453f-9b62-74c4d12cd402 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.243851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06046d9-9582-453f-9b62-74c4d12cd402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.244884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:27.245879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06046d9-9582-453f-9b62-74c4d12cd402 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.248760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95132606-6e1e-4843-bf8c-5b71ec44adc9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.254567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f0368-eba0-428a-9612-300c042f30fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.255195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f0368-eba0-428a-9612-300c042f30fe admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.255474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f0368-eba0-428a-9612-300c042f30fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.255746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f0368-eba0-428a-9612-300c042f30fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.256451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f0368-eba0-428a-9612-300c042f30fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.259321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f0368-eba0-428a-9612-300c042f30fe admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.378977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1000/1982] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/projects => generated 407 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:27.379867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06046d9-9582-453f-9b62-74c4d12cd402 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.419967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 983/1983] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:27.419967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f0368-eba0-428a-9612-300c042f30fe admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.469580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06046d9-9582-453f-9b62-74c4d12cd402 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.522743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f0368-eba0-428a-9612-300c042f30fe admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.530848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a11ae-98c5-433b-b7cb-8baf79c318fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.532115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a11ae-98c5-433b-b7cb-8baf79c318fc admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.532649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a11ae-98c5-433b-b7cb-8baf79c318fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.533123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a11ae-98c5-433b-b7cb-8baf79c318fc admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.534203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a11ae-98c5-433b-b7cb-8baf79c318fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.537807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a11ae-98c5-433b-b7cb-8baf79c318fc admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.549984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.550697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.551029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.551337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.552025 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:27.553106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.554907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.569728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1001/1984] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:27.570603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.581735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd148f79-3e5b-4e8d-a27f-7969cf5929b9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.586642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf1a51-7746-4eae-bdbb-8feee4204a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.590234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf1a51-7746-4eae-bdbb-8feee4204a9d admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.590234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf1a51-7746-4eae-bdbb-8feee4204a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.590234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf1a51-7746-4eae-bdbb-8feee4204a9d admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.590234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf1a51-7746-4eae-bdbb-8feee4204a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.590950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf1a51-7746-4eae-bdbb-8feee4204a9d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.647606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 984/1985] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/users => generated 310 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:27.648730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a11ae-98c5-433b-b7cb-8baf79c318fc admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.659892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a11ae-98c5-433b-b7cb-8baf79c318fc admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.664945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18a93d-9883-4f6d-a2a1-e544126a65ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.665570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a93d-9883-4f6d-a2a1-e544126a65ad admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.665856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a93d-9883-4f6d-a2a1-e544126a65ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.666124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a93d-9883-4f6d-a2a1-e544126a65ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.666775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18a93d-9883-4f6d-a2a1-e544126a65ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.668855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18a93d-9883-4f6d-a2a1-e544126a65ad admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.689361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1002/1986] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:27.689361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf1a51-7746-4eae-bdbb-8feee4204a9d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.705607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf1a51-7746-4eae-bdbb-8feee4204a9d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.713547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5d67f-59d6-432d-ad76-0d80c88ff179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.714526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d67f-59d6-432d-ad76-0d80c88ff179 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.715048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d67f-59d6-432d-ad76-0d80c88ff179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.715415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d67f-59d6-432d-ad76-0d80c88ff179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.716583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5d67f-59d6-432d-ad76-0d80c88ff179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.721015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5d67f-59d6-432d-ad76-0d80c88ff179 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.734410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1003/1987] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:27.735240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5d67f-59d6-432d-ad76-0d80c88ff179 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.745741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5d67f-59d6-432d-ad76-0d80c88ff179 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.752039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a0dcf-2320-42df-9602-ae6689661e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.752706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a0dcf-2320-42df-9602-ae6689661e75 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.753433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a0dcf-2320-42df-9602-ae6689661e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.753433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a0dcf-2320-42df-9602-ae6689661e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.753936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a0dcf-2320-42df-9602-ae6689661e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.755547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a0dcf-2320-42df-9602-ae6689661e75 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.767275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1004/1988] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:27.768150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a0dcf-2320-42df-9602-ae6689661e75 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.781031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a0dcf-2320-42df-9602-ae6689661e75 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.787189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df408-4d3a-449c-922c-9ed02de247fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.787189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df408-4d3a-449c-922c-9ed02de247fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.787189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df408-4d3a-449c-922c-9ed02de247fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.787189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345df408-4d3a-449c-922c-9ed02de247fb admin admin] PATH_INFO: `/v3/projects/de08a94a63e24f98b8e31609560b33d9/users/e298bb28572e422ea59f0939ae0beb9e/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.804342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345df408-4d3a-449c-922c-9ed02de247fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=de08a94a63e24f98b8e31609560b33d9, user_id=e298bb28572e422ea59f0939ae0beb9e, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.807844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 985/1989] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/projects => generated 391 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:27.808076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345df408-4d3a-449c-922c-9ed02de247fb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.808704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18a93d-9883-4f6d-a2a1-e544126a65ad admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.880681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18a93d-9883-4f6d-a2a1-e544126a65ad admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.885475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8a607-4c69-43a0-ad7f-d651a5318424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.886098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a607-4c69-43a0-ad7f-d651a5318424 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.886378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a607-4c69-43a0-ad7f-d651a5318424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.886655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8a607-4c69-43a0-ad7f-d651a5318424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.887319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8a607-4c69-43a0-ad7f-d651a5318424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.889393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb8a607-4c69-43a0-ad7f-d651a5318424 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.902692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 986/1990] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:27.903626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8a607-4c69-43a0-ad7f-d651a5318424 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:27.909254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:27.915146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8a607-4c69-43a0-ad7f-d651a5318424 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:27.920698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccc89d-63f3-4d68-ba15-b8713e59675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:27.921348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccc89d-63f3-4d68-ba15-b8713e59675c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:27.921637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccc89d-63f3-4d68-ba15-b8713e59675c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:27.921918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ccc89d-63f3-4d68-ba15-b8713e59675c admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:27.922569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccc89d-63f3-4d68-ba15-b8713e59675c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:27.924585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ccc89d-63f3-4d68-ba15-b8713e59675c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:27.942899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1005/1991] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:27 2022] PUT /identity/v3/projects/de08a94a63e24f98b8e31609560b33d9/users/e298bb28572e422ea59f0939ae0beb9e/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:27.943896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df408-4d3a-449c-922c-9ed02de247fb admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.018914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345df408-4d3a-449c-922c-9ed02de247fb admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.024315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca02f1-d806-4639-8c48-e78010667252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.027922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca02f1-d806-4639-8c48-e78010667252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.027922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca02f1-d806-4639-8c48-e78010667252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.027922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca02f1-d806-4639-8c48-e78010667252 admin admin] PATH_INFO: `/v3/projects/4b369b2aec4f48a68616cb84e3eba89d/users/6ee36b3abe114411b7c089d8ad03d0f1/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.028833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 987/1992] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:27 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:28.030959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccc89d-63f3-4d68-ba15-b8713e59675c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.043325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:28.044083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca02f1-d806-4639-8c48-e78010667252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b369b2aec4f48a68616cb84e3eba89d, user_id=6ee36b3abe114411b7c089d8ad03d0f1, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.048033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca02f1-d806-4639-8c48-e78010667252 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.048341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ccc89d-63f3-4d68-ba15-b8713e59675c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.054533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca6b3c-a00b-42ca-a892-78e447ee62c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.055052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6b3c-a00b-42ca-a892-78e447ee62c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.055332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6b3c-a00b-42ca-a892-78e447ee62c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.055611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca6b3c-a00b-42ca-a892-78e447ee62c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.056308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca6b3c-a00b-42ca-a892-78e447ee62c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.057912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca6b3c-a00b-42ca-a892-78e447ee62c4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.069989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 988/1993] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:28.071342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca6b3c-a00b-42ca-a892-78e447ee62c4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.082235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca6b3c-a00b-42ca-a892-78e447ee62c4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.086932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3817e-7f95-45ca-88c5-fffb7bcb5ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.087544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3817e-7f95-45ca-88c5-fffb7bcb5ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.087823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3817e-7f95-45ca-88c5-fffb7bcb5ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.088095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3817e-7f95-45ca-88c5-fffb7bcb5ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.088877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3817e-7f95-45ca-88c5-fffb7bcb5ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.090813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3817e-7f95-45ca-88c5-fffb7bcb5ccb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.102388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 989/1994] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:28.103291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3817e-7f95-45ca-88c5-fffb7bcb5ccb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.114418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3817e-7f95-45ca-88c5-fffb7bcb5ccb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.119184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be468a-671f-48c8-a489-ad40dbb6860b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.119853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be468a-671f-48c8-a489-ad40dbb6860b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.120131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be468a-671f-48c8-a489-ad40dbb6860b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.120435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be468a-671f-48c8-a489-ad40dbb6860b admin admin] PATH_INFO: `/v3/projects/de08a94a63e24f98b8e31609560b33d9/users/e298bb28572e422ea59f0939ae0beb9e/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.122529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be468a-671f-48c8-a489-ad40dbb6860b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de08a94a63e24f98b8e31609560b33d9, user_id=e298bb28572e422ea59f0939ae0beb9e, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.125362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be468a-671f-48c8-a489-ad40dbb6860b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.143522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 990/1995] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:28 2022] PUT /identity/v3/projects/de08a94a63e24f98b8e31609560b33d9/users/e298bb28572e422ea59f0939ae0beb9e/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:28.143522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be468a-671f-48c8-a489-ad40dbb6860b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.201657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1006/1996] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:27 2022] PUT /identity/v3/projects/4b369b2aec4f48a68616cb84e3eba89d/users/6ee36b3abe114411b7c089d8ad03d0f1/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:28.211632 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878c519-7c0b-4930-9c39-0244b6d55692 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.212313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878c519-7c0b-4930-9c39-0244b6d55692 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.212800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878c519-7c0b-4930-9c39-0244b6d55692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.275321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3878c519-7c0b-4930-9c39-0244b6d55692 None None] MFA Rules not processed for user `e298bb28572e422ea59f0939ae0beb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:28.278399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3878c519-7c0b-4930-9c39-0244b6d55692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:28.308130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be468a-671f-48c8-a489-ad40dbb6860b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.386897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1007/1997] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:28 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:28.390333 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c833ad-0d64-45eb-b06a-006e8ca4f2c6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.391500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c833ad-0d64-45eb-b06a-006e8ca4f2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.392576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c833ad-0d64-45eb-b06a-006e8ca4f2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.401933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.402567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.402848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.403289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.403971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.406494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.419306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 991/1998] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:28.420460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.422439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:28.430101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-93c833ad-0d64-45eb-b06a-006e8ca4f2c6 None None] MFA Rules not processed for user `6ee36b3abe114411b7c089d8ad03d0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:28.431855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-93c833ad-0d64-45eb-b06a-006e8ca4f2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:28.491416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42a4f2-ce3e-4352-afcb-e06ef4f70555 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.495522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bc753-cf95-4c04-8b54-53f49af62001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.496142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bc753-cf95-4c04-8b54-53f49af62001 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.496465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bc753-cf95-4c04-8b54-53f49af62001 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.496881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bc753-cf95-4c04-8b54-53f49af62001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.508167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bc753-cf95-4c04-8b54-53f49af62001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.510151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bc753-cf95-4c04-8b54-53f49af62001 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.524796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 992/1999] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:28 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:28.525076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bc753-cf95-4c04-8b54-53f49af62001 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.536705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1008/2000] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:28.538715 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bc753-cf95-4c04-8b54-53f49af62001 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.538839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c833ad-0d64-45eb-b06a-006e8ca4f2c6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.543535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ee62-d97a-4cc6-b7a3-9dfb167c6ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.544270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ee62-d97a-4cc6-b7a3-9dfb167c6ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.544534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ee62-d97a-4cc6-b7a3-9dfb167c6ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.544798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ee62-d97a-4cc6-b7a3-9dfb167c6ca3 admin admin] PATH_INFO: `/v3/projects/42caa869502e4b8798040465c59fb643/users/5b0a28db719e47239a15ba5a86f64cc7/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.552557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c833ad-0d64-45eb-b06a-006e8ca4f2c6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.559030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184fc6-c83b-4ad9-8a75-93a7de7a1e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.560141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184fc6-c83b-4ad9-8a75-93a7de7a1e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.560141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184fc6-c83b-4ad9-8a75-93a7de7a1e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.560141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184fc6-c83b-4ad9-8a75-93a7de7a1e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.560931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64184fc6-c83b-4ad9-8a75-93a7de7a1e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.564505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64184fc6-c83b-4ad9-8a75-93a7de7a1e5d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.567732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ee62-d97a-4cc6-b7a3-9dfb167c6ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42caa869502e4b8798040465c59fb643, user_id=5b0a28db719e47239a15ba5a86f64cc7, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.573521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ee62-d97a-4cc6-b7a3-9dfb167c6ca3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.696724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 993/2001] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:28 2022] PUT /identity/v3/projects/42caa869502e4b8798040465c59fb643/users/5b0a28db719e47239a15ba5a86f64cc7/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:28.709941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b136-5b91-4e18-9ab5-fc2670b4b142 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.710393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b136-5b91-4e18-9ab5-fc2670b4b142 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.710862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b136-5b91-4e18-9ab5-fc2670b4b142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.716802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:28.727724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1009/2002] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:28 2022] POST /identity/v3/projects => generated 369 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:28.733414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184fc6-c83b-4ad9-8a75-93a7de7a1e5d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.763140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4ec1b136-5b91-4e18-9ab5-fc2670b4b142 None None] MFA Rules not processed for user `5b0a28db719e47239a15ba5a86f64cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:28.763140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4ec1b136-5b91-4e18-9ab5-fc2670b4b142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:28.822423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184fc6-c83b-4ad9-8a75-93a7de7a1e5d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.828520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b7d0a-a8e1-44cc-9c8b-d4ea5c2c756f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.829175 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b7d0a-a8e1-44cc-9c8b-d4ea5c2c756f admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.829509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b7d0a-a8e1-44cc-9c8b-d4ea5c2c756f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.829693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b7d0a-a8e1-44cc-9c8b-d4ea5c2c756f admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.830357 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b7d0a-a8e1-44cc-9c8b-d4ea5c2c756f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.832471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b7d0a-a8e1-44cc-9c8b-d4ea5c2c756f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.849243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:28.853903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 994/2003] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:28.857080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1b136-5b91-4e18-9ab5-fc2670b4b142 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:28.878053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1b136-5b91-4e18-9ab5-fc2670b4b142 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:28.883779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff3c85-2062-4c27-b9a8-b0408d3eceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:28.888266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3c85-2062-4c27-b9a8-b0408d3eceb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:28.888266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3c85-2062-4c27-b9a8-b0408d3eceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:28.888266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff3c85-2062-4c27-b9a8-b0408d3eceb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:28.888266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff3c85-2062-4c27-b9a8-b0408d3eceb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:28.890389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff3c85-2062-4c27-b9a8-b0408d3eceb8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:28.988365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1010/2004] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:28 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:28.992792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b7d0a-a8e1-44cc-9c8b-d4ea5c2c756f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.003670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b7d0a-a8e1-44cc-9c8b-d4ea5c2c756f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.008847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d2b1e-bb77-4f92-a421-6917b96dc5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.009497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d2b1e-bb77-4f92-a421-6917b96dc5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.009785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d2b1e-bb77-4f92-a421-6917b96dc5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.010066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d2b1e-bb77-4f92-a421-6917b96dc5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.010800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d2b1e-bb77-4f92-a421-6917b96dc5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.012583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d2b1e-bb77-4f92-a421-6917b96dc5c1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.027177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1011/2005] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:29.029256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 995/2006] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:28 2022] POST /identity/v3/projects => generated 393 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:29.035357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff3c85-2062-4c27-b9a8-b0408d3eceb8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.036535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d2b1e-bb77-4f92-a421-6917b96dc5c1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.055502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d2b1e-bb77-4f92-a421-6917b96dc5c1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.143395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff3c85-2062-4c27-b9a8-b0408d3eceb8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.150135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da53fec-243b-48dd-a09b-0564bc5c4739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.151013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53fec-243b-48dd-a09b-0564bc5c4739 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.151495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53fec-243b-48dd-a09b-0564bc5c4739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.151982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da53fec-243b-48dd-a09b-0564bc5c4739 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.153194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da53fec-243b-48dd-a09b-0564bc5c4739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.157374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da53fec-243b-48dd-a09b-0564bc5c4739 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.165970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fe16c-0f10-4755-8c6a-f00cbc899519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.166695 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fe16c-0f10-4755-8c6a-f00cbc899519 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.166984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fe16c-0f10-4755-8c6a-f00cbc899519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.167263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fe16c-0f10-4755-8c6a-f00cbc899519 admin admin] PATH_INFO: `/v3/projects/d5d8cf72f92443b58eb8ec953fc39ae8/users/a9e85504087b488187764b03d8da8161/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.185457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fe16c-0f10-4755-8c6a-f00cbc899519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d8cf72f92443b58eb8ec953fc39ae8, user_id=a9e85504087b488187764b03d8da8161, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.190879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fe16c-0f10-4755-8c6a-f00cbc899519 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.291685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 996/2007] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:29 2022] POST /identity/v3/users => generated 303 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:29.292891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da53fec-243b-48dd-a09b-0564bc5c4739 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.341145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1012/2008] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:29 2022] PUT /identity/v3/projects/d5d8cf72f92443b58eb8ec953fc39ae8/users/a9e85504087b488187764b03d8da8161/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:29.341145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fe16c-0f10-4755-8c6a-f00cbc899519 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.361855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da53fec-243b-48dd-a09b-0564bc5c4739 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.415082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fe16c-0f10-4755-8c6a-f00cbc899519 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.422131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45de4e5-ce99-4a98-9e91-5130e86ff3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.422767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de4e5-ce99-4a98-9e91-5130e86ff3cd admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.423058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de4e5-ce99-4a98-9e91-5130e86ff3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.423335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45de4e5-ce99-4a98-9e91-5130e86ff3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.424020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45de4e5-ce99-4a98-9e91-5130e86ff3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.425702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45de4e5-ce99-4a98-9e91-5130e86ff3cd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.430919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab10c-de48-469e-b04f-b2494dbbc463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.431587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab10c-de48-469e-b04f-b2494dbbc463 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.431865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab10c-de48-469e-b04f-b2494dbbc463 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.432135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ab10c-de48-469e-b04f-b2494dbbc463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.437773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1013/2009] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:29.438726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45de4e5-ce99-4a98-9e91-5130e86ff3cd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.453706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45de4e5-ce99-4a98-9e91-5130e86ff3cd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.458922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c1485-cbe1-41c6-be89-aa4c1e5fc5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.459543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c1485-cbe1-41c6-be89-aa4c1e5fc5dd admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.466626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c1485-cbe1-41c6-be89-aa4c1e5fc5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.466626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c1485-cbe1-41c6-be89-aa4c1e5fc5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.466626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c1485-cbe1-41c6-be89-aa4c1e5fc5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.466626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c1485-cbe1-41c6-be89-aa4c1e5fc5dd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.475076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1014/2010] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:29.478858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c1485-cbe1-41c6-be89-aa4c1e5fc5dd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.490753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c1485-cbe1-41c6-be89-aa4c1e5fc5dd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.493511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:29.497837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc668cb-15c2-42fc-ab38-4283e5ab9db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.498664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc668cb-15c2-42fc-ab38-4283e5ab9db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.505100 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc668cb-15c2-42fc-ab38-4283e5ab9db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.505100 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc668cb-15c2-42fc-ab38-4283e5ab9db9 admin admin] PATH_INFO: `/v3/projects/dec7f625ca71455b9e94ee505e6c09e8/users/fccf2e571c9540beab5646d06ba8eebb/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.517166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc668cb-15c2-42fc-ab38-4283e5ab9db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec7f625ca71455b9e94ee505e6c09e8, user_id=fccf2e571c9540beab5646d06ba8eebb, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.519927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc668cb-15c2-42fc-ab38-4283e5ab9db9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.531078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab10c-de48-469e-b04f-b2494dbbc463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.532663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792ab10c-de48-469e-b04f-b2494dbbc463 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.540320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:29.546758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 997/2011] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:29 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:29.547716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab10c-de48-469e-b04f-b2494dbbc463 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.560045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ab10c-de48-469e-b04f-b2494dbbc463 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.565549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565bac48-da58-4ca7-9b9c-b815dce67fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.566340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bac48-da58-4ca7-9b9c-b815dce67fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.566735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bac48-da58-4ca7-9b9c-b815dce67fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.567094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bac48-da58-4ca7-9b9c-b815dce67fc0 admin admin] PATH_INFO: `/v3/projects/d5d8cf72f92443b58eb8ec953fc39ae8/users/a9e85504087b488187764b03d8da8161/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.569359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565bac48-da58-4ca7-9b9c-b815dce67fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d8cf72f92443b58eb8ec953fc39ae8, user_id=a9e85504087b488187764b03d8da8161, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.572224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565bac48-da58-4ca7-9b9c-b815dce67fc0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.584465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 998/2012] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:29 2022] PUT /identity/v3/projects/d5d8cf72f92443b58eb8ec953fc39ae8/users/a9e85504087b488187764b03d8da8161/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:29.597520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e0b19-a002-4150-b182-4d6ad7050b8c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.597820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e0b19-a002-4150-b182-4d6ad7050b8c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.598107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e0b19-a002-4150-b182-4d6ad7050b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.622957 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1015/2013] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:29 2022] PUT /identity/v3/projects/dec7f625ca71455b9e94ee505e6c09e8/users/fccf2e571c9540beab5646d06ba8eebb/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:29.630174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc668cb-15c2-42fc-ab38-4283e5ab9db9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.636698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a68e0b19-a002-4150-b182-4d6ad7050b8c None None] MFA Rules not processed for user `a9e85504087b488187764b03d8da8161`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:29.638426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a68e0b19-a002-4150-b182-4d6ad7050b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:29.729540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc668cb-15c2-42fc-ab38-4283e5ab9db9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.738350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74924f-c752-4f87-b146-df46dba514e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.739323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74924f-c752-4f87-b146-df46dba514e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.739772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74924f-c752-4f87-b146-df46dba514e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.740704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74924f-c752-4f87-b146-df46dba514e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.741738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74924f-c752-4f87-b146-df46dba514e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.746625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74924f-c752-4f87-b146-df46dba514e4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.754274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 999/2014] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:29 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:29.760735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1016/2015] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:29.765231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e0b19-a002-4150-b182-4d6ad7050b8c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:29.776102 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e0b19-a002-4150-b182-4d6ad7050b8c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:29.781563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690dc4a-583c-464a-b515-8c6eccc99622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:29.782233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690dc4a-583c-464a-b515-8c6eccc99622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.782509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690dc4a-583c-464a-b515-8c6eccc99622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.782783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690dc4a-583c-464a-b515-8c6eccc99622 admin admin] PATH_INFO: `/v3/projects/dec7f625ca71455b9e94ee505e6c09e8/users/fccf2e571c9540beab5646d06ba8eebb/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.784765 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690dc4a-583c-464a-b515-8c6eccc99622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec7f625ca71455b9e94ee505e6c09e8, user_id=fccf2e571c9540beab5646d06ba8eebb, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:29.787511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690dc4a-583c-464a-b515-8c6eccc99622 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:29.799346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1000/2016] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:29 2022] PUT /identity/v3/projects/dec7f625ca71455b9e94ee505e6c09e8/users/fccf2e571c9540beab5646d06ba8eebb/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:29.810480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268f241-7e4c-48cc-9121-dafa5449412b None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:29.810812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268f241-7e4c-48cc-9121-dafa5449412b None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:29.811749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268f241-7e4c-48cc-9121-dafa5449412b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:29.845689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1268f241-7e4c-48cc-9121-dafa5449412b None None] MFA Rules not processed for user `fccf2e571c9540beab5646d06ba8eebb`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:29.847092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1268f241-7e4c-48cc-9121-dafa5449412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:29.920886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:29.953082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1017/2017] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:30.302888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:30.394092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690dc4a-583c-464a-b515-8c6eccc99622 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:30.459726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690dc4a-583c-464a-b515-8c6eccc99622 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:30.459847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b561819-c262-4222-b2d4-53d3bfd2c48f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:30.460453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b561819-c262-4222-b2d4-53d3bfd2c48f service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:30.460741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b561819-c262-4222-b2d4-53d3bfd2c48f service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:30.461017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b561819-c262-4222-b2d4-53d3bfd2c48f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:30.554015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b561819-c262-4222-b2d4-53d3bfd2c48f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:30.554921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b561819-c262-4222-b2d4-53d3bfd2c48f service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:30.565849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1001/2018] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:30.621566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:30.711192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268f241-7e4c-48cc-9121-dafa5449412b None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:30.722588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268f241-7e4c-48cc-9121-dafa5449412b None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:30.729220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ba53e-a2d3-4bcb-8018-82191b1a4a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:30.729856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba53e-a2d3-4bcb-8018-82191b1a4a43 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:30.730135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba53e-a2d3-4bcb-8018-82191b1a4a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:30.730402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ba53e-a2d3-4bcb-8018-82191b1a4a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:30.740861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ba53e-a2d3-4bcb-8018-82191b1a4a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:30.742481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ba53e-a2d3-4bcb-8018-82191b1a4a43 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:30.757413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1018/2019] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:30 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:30.907560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:30.978157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:30.979224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b561819-c262-4222-b2d4-53d3bfd2c48f service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:30.988271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b561819-c262-4222-b2d4-53d3bfd2c48f service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:30.993228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484017d-3240-4691-875b-f328ca0d9545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:30.993841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484017d-3240-4691-875b-f328ca0d9545 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:30.994122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484017d-3240-4691-875b-f328ca0d9545 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:30.994394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484017d-3240-4691-875b-f328ca0d9545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:31.061106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484017d-3240-4691-875b-f328ca0d9545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:31.062999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484017d-3240-4691-875b-f328ca0d9545 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:31.073094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1002/2020] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:31.319526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:31.490676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:31.539709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:31.896244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:31.977665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:32.410978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:32.758933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:33.130574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:33.278221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:33.587461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:34.233510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:34.722470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:35.193508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:35.585624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:36.140586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:36.713308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:36.954709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:37.599021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:37.903151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ba53e-a2d3-4bcb-8018-82191b1a4a43 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:37.967599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ba53e-a2d3-4bcb-8018-82191b1a4a43 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:37.972445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:37.973087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:37.973368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:37.973756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] PATH_INFO: `/v3/users/6ee36b3abe114411b7c089d8ad03d0f1` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:37.974905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ee36b3abe114411b7c089d8ad03d0f1)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:37.976887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:37.990647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:38.162574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ee36b3abe114411b7c089d8ad03d0f1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:38.163261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ee36b3abe114411b7c089d8ad03d0f1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:38.170684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ee36b3abe114411b7c089d8ad03d0f1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:38.259430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ee36b3abe114411b7c089d8ad03d0f1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:38.260039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ee36b3abe114411b7c089d8ad03d0f1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:38.282151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1019/2021] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:37 2022] DELETE /identity/v3/users/6ee36b3abe114411b7c089d8ad03d0f1 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:38.286907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32be62-f93c-430e-b91e-4b846a4a451c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:38.287371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32be62-f93c-430e-b91e-4b846a4a451c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:38.287487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32be62-f93c-430e-b91e-4b846a4a451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:38.307310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7e32be62-f93c-430e-b91e-4b846a4a451c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:38.308834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7e32be62-f93c-430e-b91e-4b846a4a451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:38.365919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:38.376011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1003/2022] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:38.387211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:38.389684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:38.450507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7546893-4efa-42e3-9d4f-6a1398495506 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:38.454950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c852503-cda3-4ab0-9946-1ae02ef3dec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:38.455556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c852503-cda3-4ab0-9946-1ae02ef3dec7 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:38.455836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c852503-cda3-4ab0-9946-1ae02ef3dec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:38.456110 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c852503-cda3-4ab0-9946-1ae02ef3dec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:38.466319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c852503-cda3-4ab0-9946-1ae02ef3dec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:38.467793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c852503-cda3-4ab0-9946-1ae02ef3dec7 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:38.478369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1020/2023] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:38.490369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:38.783165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:39.048037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32be62-f93c-430e-b91e-4b846a4a451c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:39.049598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7e32be62-f93c-430e-b91e-4b846a4a451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:39.071051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32be62-f93c-430e-b91e-4b846a4a451c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:39.077109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:39.077756 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:39.078910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:39.078910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] PATH_INFO: `/v3/projects/4b369b2aec4f48a68616cb84e3eba89d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:39.079524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b369b2aec4f48a68616cb84e3eba89d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:39.081503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:39.105945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:39.196433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b369b2aec4f48a68616cb84e3eba89d', 'request_id': 'req-556bbb6c-4619-4a6e-9106-0f255ddeddd8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.202945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b369b2aec4f48a68616cb84e3eba89d', 'request_id': 'req-556bbb6c-4619-4a6e-9106-0f255ddeddd8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.210432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b369b2aec4f48a68616cb84e3eba89d', 'request_id': 'req-556bbb6c-4619-4a6e-9106-0f255ddeddd8'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.282726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] The token cache is being invalidate because project 4b369b2aec4f48a68616cb84e3eba89d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:39.283018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.284418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.286022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1004/2024] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:39 2022] DELETE /identity/v3/projects/4b369b2aec4f48a68616cb84e3eba89d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:39.290080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c852503-cda3-4ab0-9946-1ae02ef3dec7 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:39.300546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c852503-cda3-4ab0-9946-1ae02ef3dec7 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:39.333947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:39.368112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:39.368824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:39.369109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:39.369393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] PATH_INFO: `/v3/users/a9e85504087b488187764b03d8da8161` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:39.370541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e85504087b488187764b03d8da8161)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:39.372527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:39.577084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e85504087b488187764b03d8da8161'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.577780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e85504087b488187764b03d8da8161'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.586756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e85504087b488187764b03d8da8161'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.734554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e85504087b488187764b03d8da8161'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.735899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f792f464-4ca8-4a3a-9b28-7895a38ffd7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e85504087b488187764b03d8da8161'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:39.756136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1021/2025] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:39 2022] DELETE /identity/v3/users/a9e85504087b488187764b03d8da8161 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:40.241164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:40.242613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:40.262798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bbb6c-4619-4a6e-9106-0f255ddeddd8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:40.330125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:40.330646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:40.330926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:40.331201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] PATH_INFO: `/v3/projects/d5d8cf72f92443b58eb8ec953fc39ae8` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:40.332481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d8cf72f92443b58eb8ec953fc39ae8)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:40.334532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:40.349212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:40.485227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d8cf72f92443b58eb8ec953fc39ae8', 'request_id': 'req-c44f7ee1-af78-4b08-9d76-72803f2d528e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:40.490755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d8cf72f92443b58eb8ec953fc39ae8', 'request_id': 'req-c44f7ee1-af78-4b08-9d76-72803f2d528e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:40.502997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d8cf72f92443b58eb8ec953fc39ae8', 'request_id': 'req-c44f7ee1-af78-4b08-9d76-72803f2d528e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:40.716255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] The token cache is being invalidate because project d5d8cf72f92443b58eb8ec953fc39ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:40.716999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:40.717961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f7ee1-af78-4b08-9d76-72803f2d528e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:40.719727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1005/2026] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:40 2022] DELETE /identity/v3/projects/d5d8cf72f92443b58eb8ec953fc39ae8 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:40.724706 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4e5d2-0e40-43bf-a689-0b48ed408712 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:40.725265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4e5d2-0e40-43bf-a689-0b48ed408712 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:40.725265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4e5d2-0e40-43bf-a689-0b48ed408712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:40.748991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0dd4e5d2-0e40-43bf-a689-0b48ed408712 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:40.750463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0dd4e5d2-0e40-43bf-a689-0b48ed408712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:40.818249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1022/2027] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:40.826467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebf151-c7c4-492d-a41f-0d056392f3a9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:40.826842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebf151-c7c4-492d-a41f-0d056392f3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:40.827122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebf151-c7c4-492d-a41f-0d056392f3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:40.852581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-51ebf151-c7c4-492d-a41f-0d056392f3a9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:40.853770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-51ebf151-c7c4-492d-a41f-0d056392f3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:40.863007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1006/2028] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:40.867575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4e5d2-0e40-43bf-a689-0b48ed408712 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:40.879346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4e5d2-0e40-43bf-a689-0b48ed408712 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:40.884543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:40.885166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:40.885452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:40.885730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:40.886391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:40.886700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:40.888523 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:40.902249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1023/2029] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:40.907289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebf151-c7c4-492d-a41f-0d056392f3a9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:40.918448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebf151-c7c4-492d-a41f-0d056392f3a9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:40.924693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:40.925342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:40.925623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:40.925894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:40.926541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:40.928501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:41.044428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1007/2030] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:40 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:41.051027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:41.063835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ac65c-6c07-4cf4-b5c1-273b663f320f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:41.117461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:41.161743 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3604b4-5e91-483d-8c37-09468cbd0b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:41.162448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3604b4-5e91-483d-8c37-09468cbd0b43 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.163121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3604b4-5e91-483d-8c37-09468cbd0b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.163431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3604b4-5e91-483d-8c37-09468cbd0b43 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.164161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3604b4-5e91-483d-8c37-09468cbd0b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:41.166411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3604b4-5e91-483d-8c37-09468cbd0b43 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:41.199159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:41.206284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:41.216101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06348d-a1c1-4378-ad0b-2f88f9e2b68c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:41.222895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:41.223543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.223828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.224107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] PATH_INFO: `/v3/users/5b0a28db719e47239a15ba5a86f64cc7` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.225400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b0a28db719e47239a15ba5a86f64cc7)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:41.227370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:41.431795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1024/2031] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:41 2022] POST /identity/v3/users => generated 284 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:41.439378 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3604b4-5e91-483d-8c37-09468cbd0b43 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:41.460124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3604b4-5e91-483d-8c37-09468cbd0b43 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:41.470317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c1a0d-2dfc-452c-bc05-fe4df5c77a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:41.471335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c1a0d-2dfc-452c-bc05-fe4df5c77a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.471837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c1a0d-2dfc-452c-bc05-fe4df5c77a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.472375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c1a0d-2dfc-452c-bc05-fe4df5c77a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.473375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c1a0d-2dfc-452c-bc05-fe4df5c77a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:41.478723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c1a0d-2dfc-452c-bc05-fe4df5c77a69 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:41.509457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1025/2032] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:41.511151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c1a0d-2dfc-452c-bc05-fe4df5c77a69 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:41.533968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b0a28db719e47239a15ba5a86f64cc7'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:41.535684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b0a28db719e47239a15ba5a86f64cc7'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:41.538453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c1a0d-2dfc-452c-bc05-fe4df5c77a69 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:41.543640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b0a28db719e47239a15ba5a86f64cc7'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:41.544372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b0a28db719e47239a15ba5a86f64cc7'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:41.546320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:41.546936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.547227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.547506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.548207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:41.550090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:41.562275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1026/2033] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:41.566779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:41.568731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:41.582568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f757-95c1-470f-bf4a-b75dc963cb65 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:41.587617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4097b9-7df3-465b-9de7-8a602c3673bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:41.588322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4097b9-7df3-465b-9de7-8a602c3673bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.588606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4097b9-7df3-465b-9de7-8a602c3673bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.588878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4097b9-7df3-465b-9de7-8a602c3673bb admin admin] PATH_INFO: `/v3/projects/e541ee3b3ecd432baee0983f51dc05de/users/090c8775b66946a6a0ae0ddd8560df3b/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.605174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4097b9-7df3-465b-9de7-8a602c3673bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e541ee3b3ecd432baee0983f51dc05de, user_id=090c8775b66946a6a0ae0ddd8560df3b, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:41.608694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4097b9-7df3-465b-9de7-8a602c3673bb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:41.661542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-03759631-4631-49aa-b7da-2e3582c192f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b0a28db719e47239a15ba5a86f64cc7'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:41.680647 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1008/2034] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:41 2022] DELETE /identity/v3/users/5b0a28db719e47239a15ba5a86f64cc7 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:41.685970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b137b-8f4b-4565-93d3-9bd5cd20479c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.686256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b137b-8f4b-4565-93d3-9bd5cd20479c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.686538 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b137b-8f4b-4565-93d3-9bd5cd20479c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.706013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-258b137b-8f4b-4565-93d3-9bd5cd20479c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:41.713107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-258b137b-8f4b-4565-93d3-9bd5cd20479c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:41.713264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1027/2035] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:41 2022] PUT /identity/v3/projects/e541ee3b3ecd432baee0983f51dc05de/users/090c8775b66946a6a0ae0ddd8560df3b/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:41.723941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b6fb9-9953-46da-b48b-1737b67b5dd6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.724607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b6fb9-9953-46da-b48b-1737b67b5dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.724991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b6fb9-9953-46da-b48b-1737b67b5dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.761059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7c1b6fb9-9953-46da-b48b-1737b67b5dd6 None None] MFA Rules not processed for user `090c8775b66946a6a0ae0ddd8560df3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:41.764589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7c1b6fb9-9953-46da-b48b-1737b67b5dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:41.765694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:41.785277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1009/2036] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:41.797333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b137b-8f4b-4565-93d3-9bd5cd20479c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:41.799479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-258b137b-8f4b-4565-93d3-9bd5cd20479c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:41.840446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1028/2037] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:41.845927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b6fb9-9953-46da-b48b-1737b67b5dd6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:41.858380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b6fb9-9953-46da-b48b-1737b67b5dd6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:41.863061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c83b7-62ab-4467-a2b2-97db13e3f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:41.863684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c83b7-62ab-4467-a2b2-97db13e3f1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.863962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c83b7-62ab-4467-a2b2-97db13e3f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.864269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c83b7-62ab-4467-a2b2-97db13e3f1b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.864909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c83b7-62ab-4467-a2b2-97db13e3f1b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:41.866909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c83b7-62ab-4467-a2b2-97db13e3f1b4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:41.872398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b137b-8f4b-4565-93d3-9bd5cd20479c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:41.877118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7762c8-c314-4a60-8c98-123bcfcc9262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:41.877870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7762c8-c314-4a60-8c98-123bcfcc9262 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:41.878159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7762c8-c314-4a60-8c98-123bcfcc9262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:41.878433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7762c8-c314-4a60-8c98-123bcfcc9262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:41.888559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7762c8-c314-4a60-8c98-123bcfcc9262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:41.890024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7762c8-c314-4a60-8c98-123bcfcc9262 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:41.900879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1010/2038] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:41.974202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1029/2039] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:41 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:41.978172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7762c8-c314-4a60-8c98-123bcfcc9262 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:41.990149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7762c8-c314-4a60-8c98-123bcfcc9262 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:42.056565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f80f3-834d-4345-bbb6-dced7c03f097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:42.057194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f80f3-834d-4345-bbb6-dced7c03f097 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.057456 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f80f3-834d-4345-bbb6-dced7c03f097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.057727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f80f3-834d-4345-bbb6-dced7c03f097 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.058619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f80f3-834d-4345-bbb6-dced7c03f097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:42.060990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f80f3-834d-4345-bbb6-dced7c03f097 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:42.177034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1011/2040] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:41 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:42.181123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c83b7-62ab-4467-a2b2-97db13e3f1b4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:42.192522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c83b7-62ab-4467-a2b2-97db13e3f1b4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:42.197151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edf54e-4037-4b73-ae8b-64e0dc48d959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:42.197770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf54e-4037-4b73-ae8b-64e0dc48d959 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.198054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf54e-4037-4b73-ae8b-64e0dc48d959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.200091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edf54e-4037-4b73-ae8b-64e0dc48d959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.200091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edf54e-4037-4b73-ae8b-64e0dc48d959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:42.201221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edf54e-4037-4b73-ae8b-64e0dc48d959 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:42.212263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1030/2041] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:42.216761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f80f3-834d-4345-bbb6-dced7c03f097 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:42.228038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f80f3-834d-4345-bbb6-dced7c03f097 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:42.233722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:42.234395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.234680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.235160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] PATH_INFO: `/v3/projects/390262b75b5c4c6db683617c5bbe00db/users/8bda4b50a7104bde860e6f3f86a32429/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.252364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=390262b75b5c4c6db683617c5bbe00db, user_id=8bda4b50a7104bde860e6f3f86a32429, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:42.254956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:42.351397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1012/2042] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:42 2022] PUT /identity/v3/projects/390262b75b5c4c6db683617c5bbe00db/users/8bda4b50a7104bde860e6f3f86a32429/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:42.355625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edf54e-4037-4b73-ae8b-64e0dc48d959 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:42.368422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edf54e-4037-4b73-ae8b-64e0dc48d959 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:42.402920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:42.423106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:42.425225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:42.434731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b200c-7eb9-4a29-b134-8e14488bd1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:42.435517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b200c-7eb9-4a29-b134-8e14488bd1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.435797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b200c-7eb9-4a29-b134-8e14488bd1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.436153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b200c-7eb9-4a29-b134-8e14488bd1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.436754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b200c-7eb9-4a29-b134-8e14488bd1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:42.439376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b200c-7eb9-4a29-b134-8e14488bd1e2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:42.453536 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1031/2043] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:42.457596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b200c-7eb9-4a29-b134-8e14488bd1e2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:42.468502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b200c-7eb9-4a29-b134-8e14488bd1e2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:42.473538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc6a36-94ab-44a4-83db-5b4aaf32d61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:42.474205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6a36-94ab-44a4-83db-5b4aaf32d61f admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.474490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6a36-94ab-44a4-83db-5b4aaf32d61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.474771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc6a36-94ab-44a4-83db-5b4aaf32d61f admin admin] PATH_INFO: `/v3/projects/390262b75b5c4c6db683617c5bbe00db/users/8bda4b50a7104bde860e6f3f86a32429/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.478950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc6a36-94ab-44a4-83db-5b4aaf32d61f admin admin] RBAC: Authorizing `identity:create_grant(project_id=390262b75b5c4c6db683617c5bbe00db, user_id=8bda4b50a7104bde860e6f3f86a32429, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:42.482850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc6a36-94ab-44a4-83db-5b4aaf32d61f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:42.495647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1032/2044] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:42 2022] PUT /identity/v3/projects/390262b75b5c4c6db683617c5bbe00db/users/8bda4b50a7104bde860e6f3f86a32429/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:42.506883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2aff3-88dc-463a-a06c-7fd6f1b9b57d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.507398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2aff3-88dc-463a-a06c-7fd6f1b9b57d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.507898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2aff3-88dc-463a-a06c-7fd6f1b9b57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.512359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:42.538805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc47669-1850-4388-9ed5-ce7ca4c6f052 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:42.559517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8bb2aff3-88dc-463a-a06c-7fd6f1b9b57d None None] MFA Rules not processed for user `8bda4b50a7104bde860e6f3f86a32429`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:42.563001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8bb2aff3-88dc-463a-a06c-7fd6f1b9b57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:42.637106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:42.637747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.638032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.638949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] PATH_INFO: `/v3/projects/42caa869502e4b8798040465c59fb643` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.640660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42caa869502e4b8798040465c59fb643)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:42.642620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:42.670556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1033/2045] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:42.686070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2aff3-88dc-463a-a06c-7fd6f1b9b57d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:42.687689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8bb2aff3-88dc-463a-a06c-7fd6f1b9b57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:42.759976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2aff3-88dc-463a-a06c-7fd6f1b9b57d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:42.791958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42caa869502e4b8798040465c59fb643', 'request_id': 'req-43a98bfc-eb55-4b3f-af08-53f2371e8613'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:42.796595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42caa869502e4b8798040465c59fb643', 'request_id': 'req-43a98bfc-eb55-4b3f-af08-53f2371e8613'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:42.803444 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42caa869502e4b8798040465c59fb643', 'request_id': 'req-43a98bfc-eb55-4b3f-af08-53f2371e8613'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:42.847687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:42.848371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.848657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.848934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.865747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:42.895359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] The token cache is being invalidate because project 42caa869502e4b8798040465c59fb643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:42.895809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:42.896500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:42.898055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1013/2046] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:42 2022] DELETE /identity/v3/projects/42caa869502e4b8798040465c59fb643 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:42.902817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:42.905149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:42.917058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:42.918510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:42.929856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1034/2047] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:42.978812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98bfc-eb55-4b3f-af08-53f2371e8613 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:42.985775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:42.986007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:42.986698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:42.986698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] PATH_INFO: `/v3/users/fccf2e571c9540beab5646d06ba8eebb` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:42.988482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fccf2e571c9540beab5646d06ba8eebb)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:42.989819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:43.035275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:43.036773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:43.048142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0556b-8e53-4122-bf85-f3829be9648f service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:43.052281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:43.059116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:43.059116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:43.059116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:43.059116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:43.126173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:43.127840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:43.137053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1035/2048] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:43.183311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:43.197415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fccf2e571c9540beab5646d06ba8eebb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:43.198125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fccf2e571c9540beab5646d06ba8eebb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:43.207729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fccf2e571c9540beab5646d06ba8eebb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:43.208732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fccf2e571c9540beab5646d06ba8eebb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:43.324489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6849549c-59c0-4b3b-9d54-2a0ea1ea3841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fccf2e571c9540beab5646d06ba8eebb'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:43.345218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1014/2049] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:42 2022] DELETE /identity/v3/users/fccf2e571c9540beab5646d06ba8eebb => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:43.850532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:43.860296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45341a1a-8ca3-4d0b-af78-df0403b5edc2 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:43.925131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:43.925752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:43.926032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:43.926307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] PATH_INFO: `/v3/projects/dec7f625ca71455b9e94ee505e6c09e8` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:43.927438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec7f625ca71455b9e94ee505e6c09e8)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:43.929510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:44.049353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec7f625ca71455b9e94ee505e6c09e8', 'request_id': 'req-c691ff42-05cf-424f-9409-b3742ec406a3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:44.050967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:44.056225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec7f625ca71455b9e94ee505e6c09e8', 'request_id': 'req-c691ff42-05cf-424f-9409-b3742ec406a3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:44.187465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec7f625ca71455b9e94ee505e6c09e8', 'request_id': 'req-c691ff42-05cf-424f-9409-b3742ec406a3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:44.199589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] The token cache is being invalidate because project dec7f625ca71455b9e94ee505e6c09e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:44.199890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:44.200987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c691ff42-05cf-424f-9409-b3742ec406a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:44.204278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1036/2050] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:43 2022] DELETE /identity/v3/projects/dec7f625ca71455b9e94ee505e6c09e8 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:44.209320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f0d1-eb19-43bb-9ad8-cdb5ca327984 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.209883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f0d1-eb19-43bb-9ad8-cdb5ca327984 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.210032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819f0d1-eb19-43bb-9ad8-cdb5ca327984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.242734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8819f0d1-eb19-43bb-9ad8-cdb5ca327984 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:44.244006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8819f0d1-eb19-43bb-9ad8-cdb5ca327984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:44.309270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1015/2051] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:44.319039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1dbe7-8b03-4521-95c8-0a8fa32e7185 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.319321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1dbe7-8b03-4521-95c8-0a8fa32e7185 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.319606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1dbe7-8b03-4521-95c8-0a8fa32e7185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.339916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cdd1dbe7-8b03-4521-95c8-0a8fa32e7185 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:44.341156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cdd1dbe7-8b03-4521-95c8-0a8fa32e7185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:44.349791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1037/2052] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:44.354400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f0d1-eb19-43bb-9ad8-cdb5ca327984 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:44.366001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819f0d1-eb19-43bb-9ad8-cdb5ca327984 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:44.370852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:44.371396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.371681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.371957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.373148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:44.373665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:44.375276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:44.387506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1016/2053] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:39:44.392417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1dbe7-8b03-4521-95c8-0a8fa32e7185 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:44.405628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1dbe7-8b03-4521-95c8-0a8fa32e7185 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:44.410299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fea86-f31a-4bad-a859-2c86382fb31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:44.410890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fea86-f31a-4bad-a859-2c86382fb31f admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.411185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fea86-f31a-4bad-a859-2c86382fb31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.411440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84fea86-f31a-4bad-a859-2c86382fb31f admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.412071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fea86-f31a-4bad-a859-2c86382fb31f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:44.414031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84fea86-f31a-4bad-a859-2c86382fb31f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:44.546626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1038/2054] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:44 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:44.551132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:44.561751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66252ca6-649e-4751-b266-8fc0f89452f1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:44.618674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d358f6-53a2-494d-8805-05a012b716f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:44.619286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d358f6-53a2-494d-8805-05a012b716f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.619564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d358f6-53a2-494d-8805-05a012b716f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.619919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d358f6-53a2-494d-8805-05a012b716f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.620637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d358f6-53a2-494d-8805-05a012b716f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:44.622661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d358f6-53a2-494d-8805-05a012b716f5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:44.726057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1017/2055] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:44 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:44.727302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fea86-f31a-4bad-a859-2c86382fb31f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:44.738504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84fea86-f31a-4bad-a859-2c86382fb31f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:44.743297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff848e8f-957c-4b68-b065-fc107c57f4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:44.748666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff848e8f-957c-4b68-b065-fc107c57f4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.748666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff848e8f-957c-4b68-b065-fc107c57f4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.748666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff848e8f-957c-4b68-b065-fc107c57f4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.748666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff848e8f-957c-4b68-b065-fc107c57f4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:44.748666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff848e8f-957c-4b68-b065-fc107c57f4f6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:44.760163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1039/2056] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:44.764918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d358f6-53a2-494d-8805-05a012b716f5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:44.775723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d358f6-53a2-494d-8805-05a012b716f5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:44.781622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c3aa-5ce5-416c-9206-8368c564d654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:44.782249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c3aa-5ce5-416c-9206-8368c564d654 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.782527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c3aa-5ce5-416c-9206-8368c564d654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.782817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c3aa-5ce5-416c-9206-8368c564d654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.783469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0c3aa-5ce5-416c-9206-8368c564d654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:44.785202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0c3aa-5ce5-416c-9206-8368c564d654 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:44.799587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1018/2057] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:44.803987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff848e8f-957c-4b68-b065-fc107c57f4f6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:44.816055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff848e8f-957c-4b68-b065-fc107c57f4f6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:44.821153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:44.821839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.822122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.822400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] PATH_INFO: `/v3/projects/c22fb394e99345d88b2e082b22902af3/users/e9c7ecfdb9904095b122bb8fd82121fe/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.839874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22fb394e99345d88b2e082b22902af3, user_id=e9c7ecfdb9904095b122bb8fd82121fe, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:44.842715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:44.940976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1040/2058] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:44 2022] PUT /identity/v3/projects/c22fb394e99345d88b2e082b22902af3/users/e9c7ecfdb9904095b122bb8fd82121fe/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:44.946281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c326f-9761-442e-849d-d58f22efbb8e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:44.946574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c326f-9761-442e-849d-d58f22efbb8e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:44.946848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c326f-9761-442e-849d-d58f22efbb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:44.966973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e44c326f-9761-442e-849d-d58f22efbb8e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:44.968282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e44c326f-9761-442e-849d-d58f22efbb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:44.977950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:45.034078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1019/2059] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:45.044369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:45.046538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:45.106492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78023193-577e-4a6f-9010-49982603bc30 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:45.110458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:45.111195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:45.111517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:45.111795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:45.122323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:45.124732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:45.137171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1041/2060] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:46.711800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567546ca-0d12-4198-9cb0-c0da1b0d6a6b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:46.712420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567546ca-0d12-4198-9cb0-c0da1b0d6a6b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:46.712420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567546ca-0d12-4198-9cb0-c0da1b0d6a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:46.743602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-567546ca-0d12-4198-9cb0-c0da1b0d6a6b None None] MFA Rules not processed for user `e9c7ecfdb9904095b122bb8fd82121fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:46.744857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-567546ca-0d12-4198-9cb0-c0da1b0d6a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:46.830212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1020/2061] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:46.845549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:46.847036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:46.906133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe148b-e9dc-4ad1-b14e-8757023eeae0 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:46.910017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2ab01-a843-4b34-8be8-d36d8316871b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:46.910633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2ab01-a843-4b34-8be8-d36d8316871b service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:46.910912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2ab01-a843-4b34-8be8-d36d8316871b service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:46.911215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2ab01-a843-4b34-8be8-d36d8316871b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:46.920307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2ab01-a843-4b34-8be8-d36d8316871b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:46.921777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2ab01-a843-4b34-8be8-d36d8316871b service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:46.931161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1042/2062] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:48.675631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567546ca-0d12-4198-9cb0-c0da1b0d6a6b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:48.677252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-567546ca-0d12-4198-9cb0-c0da1b0d6a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:48.692103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567546ca-0d12-4198-9cb0-c0da1b0d6a6b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:48.696816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:48.697687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:48.697921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:48.698204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] PATH_INFO: `/v3/users/090c8775b66946a6a0ae0ddd8560df3b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:48.699367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=090c8775b66946a6a0ae0ddd8560df3b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:48.702510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:48.906682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090c8775b66946a6a0ae0ddd8560df3b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:48.907283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090c8775b66946a6a0ae0ddd8560df3b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:48.914524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090c8775b66946a6a0ae0ddd8560df3b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:48.915227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090c8775b66946a6a0ae0ddd8560df3b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:49.047672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090c8775b66946a6a0ae0ddd8560df3b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:49.073266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1021/2063] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:48 2022] DELETE /identity/v3/users/090c8775b66946a6a0ae0ddd8560df3b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:49.078547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65effb96-5f2e-4163-be12-f454b8a7db0c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:49.084776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65effb96-5f2e-4163-be12-f454b8a7db0c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:49.084776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65effb96-5f2e-4163-be12-f454b8a7db0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:49.102751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-65effb96-5f2e-4163-be12-f454b8a7db0c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:49.104244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-65effb96-5f2e-4163-be12-f454b8a7db0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:49.119257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:49.177794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1043/2064] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:49.190018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:49.191981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:49.228266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:49.298808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29afde5-202b-4e92-99ed-aff7005c9839 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:49.302854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed7281-2a60-4e1e-8551-80836c0c02d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:49.303718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7281-2a60-4e1e-8551-80836c0c02d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:49.304144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7281-2a60-4e1e-8551-80836c0c02d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:49.304461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7281-2a60-4e1e-8551-80836c0c02d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:49.314897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed7281-2a60-4e1e-8551-80836c0c02d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:49.316420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed7281-2a60-4e1e-8551-80836c0c02d6 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:49.326451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1022/2065] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:49.731750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65effb96-5f2e-4163-be12-f454b8a7db0c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:49.733382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-65effb96-5f2e-4163-be12-f454b8a7db0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:49.754501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65effb96-5f2e-4163-be12-f454b8a7db0c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:49.761172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:49.761902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:49.762117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:49.762379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] PATH_INFO: `/v3/projects/e541ee3b3ecd432baee0983f51dc05de` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:49.764676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e541ee3b3ecd432baee0983f51dc05de)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:49.768741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:49.856637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b65b4-af92-40a7-a6f6-54b2040ff969 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:49.856637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b65b4-af92-40a7-a6f6-54b2040ff969 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:49.856637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b65b4-af92-40a7-a6f6-54b2040ff969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:49.865985 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8e7b65b4-af92-40a7-a6f6-54b2040ff969 None None] MFA Rules not processed for user `e9c7ecfdb9904095b122bb8fd82121fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:49.867409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8e7b65b4-af92-40a7-a6f6-54b2040ff969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:49.925591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1023/2066] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:50.042762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e541ee3b3ecd432baee0983f51dc05de', 'request_id': 'req-f0f07bd6-c393-4bba-8abd-563461dc41b8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.055250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e541ee3b3ecd432baee0983f51dc05de', 'request_id': 'req-f0f07bd6-c393-4bba-8abd-563461dc41b8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.159036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e541ee3b3ecd432baee0983f51dc05de', 'request_id': 'req-f0f07bd6-c393-4bba-8abd-563461dc41b8'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.178041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] The token cache is being invalidate because project e541ee3b3ecd432baee0983f51dc05de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:50.178352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.179186 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.184013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1044/2067] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:49 2022] DELETE /identity/v3/projects/e541ee3b3ecd432baee0983f51dc05de => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:50.187501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b65b4-af92-40a7-a6f6-54b2040ff969 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:50.201575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b65b4-af92-40a7-a6f6-54b2040ff969 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:50.274098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:50.274981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:50.275323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:50.275789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] PATH_INFO: `/v3/users/8bda4b50a7104bde860e6f3f86a32429` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:50.277299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bda4b50a7104bde860e6f3f86a32429)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:50.279793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:50.503683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bda4b50a7104bde860e6f3f86a32429'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.504486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bda4b50a7104bde860e6f3f86a32429'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.511654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bda4b50a7104bde860e6f3f86a32429'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.512366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bda4b50a7104bde860e6f3f86a32429'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.578656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bda4b50a7104bde860e6f3f86a32429'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:50.583884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:50.598409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1024/2068] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:50 2022] DELETE /identity/v3/users/8bda4b50a7104bde860e6f3f86a32429 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:50.963553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:50.965606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:50.986338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f07bd6-c393-4bba-8abd-563461dc41b8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:51.073220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:51.073863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:51.074141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:51.074416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] PATH_INFO: `/v3/users/c5fdcaed01a74ef0ac9279d6007a3283` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:51.075712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5fdcaed01a74ef0ac9279d6007a3283)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:51.078472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:51.117362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:51.119024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:51.137256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dac10c-7164-4be1-83ed-d696fbf3063b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:51.155213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:51.156114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:51.156450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:51.156775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] PATH_INFO: `/v3/projects/390262b75b5c4c6db683617c5bbe00db` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:51.158157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=390262b75b5c4c6db683617c5bbe00db)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:51.160488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:51.393639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:51.397798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5fdcaed01a74ef0ac9279d6007a3283'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.416901 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5fdcaed01a74ef0ac9279d6007a3283'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.429619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5fdcaed01a74ef0ac9279d6007a3283'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.432545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '390262b75b5c4c6db683617c5bbe00db', 'request_id': 'req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.441378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '390262b75b5c4c6db683617c5bbe00db', 'request_id': 'req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.450810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '390262b75b5c4c6db683617c5bbe00db', 'request_id': 'req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.550275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5fdcaed01a74ef0ac9279d6007a3283'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.551316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9544b232-ce1b-4829-a029-5a9e3e80e4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5fdcaed01a74ef0ac9279d6007a3283'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.582344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1045/2069] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:50 2022] DELETE /identity/v3/users/c5fdcaed01a74ef0ac9279d6007a3283 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:51.587602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e70f1-923c-4875-ae70-e7ffd2f29825 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:51.588062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e70f1-923c-4875-ae70-e7ffd2f29825 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:51.588597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e70f1-923c-4875-ae70-e7ffd2f29825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:51.611406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c26e70f1-923c-4875-ae70-e7ffd2f29825 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:51.613066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c26e70f1-923c-4875-ae70-e7ffd2f29825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:51.675645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] The token cache is being invalidate because project 390262b75b5c4c6db683617c5bbe00db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:51.676448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.678142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b3b67efe-a457-4f4f-a7f7-bc2467c0d1ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:51.682378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1025/2070] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:51 2022] DELETE /identity/v3/projects/390262b75b5c4c6db683617c5bbe00db => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:51.689308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f9490-df24-4e22-853d-b4b523c3722f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:51.689816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f9490-df24-4e22-853d-b4b523c3722f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:51.690235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f9490-df24-4e22-853d-b4b523c3722f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:51.724526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1046/2071] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:51.730670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cd1f9490-df24-4e22-853d-b4b523c3722f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:51.732252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cd1f9490-df24-4e22-853d-b4b523c3722f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:51.744979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e70f1-923c-4875-ae70-e7ffd2f29825 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:51.749160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c26e70f1-923c-4875-ae70-e7ffd2f29825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:51.774540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1026/2072] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:51.778363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95f33-0123-4785-9b61-22638a8b7c05 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:51.784060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95f33-0123-4785-9b61-22638a8b7c05 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:51.784060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95f33-0123-4785-9b61-22638a8b7c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:51.826059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-38c95f33-0123-4785-9b61-22638a8b7c05 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:51.827272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-38c95f33-0123-4785-9b61-22638a8b7c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:51.840455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1027/2073] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:51.845925 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c95f33-0123-4785-9b61-22638a8b7c05 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:51.864140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c95f33-0123-4785-9b61-22638a8b7c05 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:51.871037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:51.871951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:51.872272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:51.872813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:51.873498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:51.873846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:51.875897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:51.882857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e70f1-923c-4875-ae70-e7ffd2f29825 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:51.889680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf797316-3b71-421b-a15b-9c6fe68c70eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:51.891231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf797316-3b71-421b-a15b-9c6fe68c70eb service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:51.891535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf797316-3b71-421b-a15b-9c6fe68c70eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:51.891983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf797316-3b71-421b-a15b-9c6fe68c70eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:51.903548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf797316-3b71-421b-a15b-9c6fe68c70eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:51.905135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf797316-3b71-421b-a15b-9c6fe68c70eb service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:51.916159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1028/2074] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:51.917490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:51.920652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1047/2075] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:51.931910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6e953-645d-455a-bb50-8bb73a9b9861 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:51.939859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c86d0-6ced-4a53-a6f0-b664057f1c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:51.940597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c86d0-6ced-4a53-a6f0-b664057f1c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:51.940880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c86d0-6ced-4a53-a6f0-b664057f1c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:51.941156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c86d0-6ced-4a53-a6f0-b664057f1c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:51.941877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c86d0-6ced-4a53-a6f0-b664057f1c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:51.943747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c86d0-6ced-4a53-a6f0-b664057f1c13 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:52.118322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1029/2076] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:51 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:52.119775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf797316-3b71-421b-a15b-9c6fe68c70eb service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:52.140028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf797316-3b71-421b-a15b-9c6fe68c70eb service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:52.229106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3533c-5478-4cce-aea5-4bd55fafa52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:52.229766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3533c-5478-4cce-aea5-4bd55fafa52a admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:52.230125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3533c-5478-4cce-aea5-4bd55fafa52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:52.230413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3533c-5478-4cce-aea5-4bd55fafa52a admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:52.231068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3533c-5478-4cce-aea5-4bd55fafa52a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:52.233045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3533c-5478-4cce-aea5-4bd55fafa52a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:52.351773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1048/2077] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:52 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:52.355912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c86d0-6ced-4a53-a6f0-b664057f1c13 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:52.367519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c86d0-6ced-4a53-a6f0-b664057f1c13 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:52.372374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79251c-963b-4c50-a7b7-65b2593eb6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:52.373056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79251c-963b-4c50-a7b7-65b2593eb6ab admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:52.373344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79251c-963b-4c50-a7b7-65b2593eb6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:52.373618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79251c-963b-4c50-a7b7-65b2593eb6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:52.374387 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79251c-963b-4c50-a7b7-65b2593eb6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:52.376391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79251c-963b-4c50-a7b7-65b2593eb6ab admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:52.389693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1030/2078] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:52.393882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3533c-5478-4cce-aea5-4bd55fafa52a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:52.404352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3533c-5478-4cce-aea5-4bd55fafa52a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:52.409667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:52.410265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:52.410550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:52.410832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:52.411487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:52.413150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:52.433167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1049/2079] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:52.436208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79251c-963b-4c50-a7b7-65b2593eb6ab admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:52.454981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79251c-963b-4c50-a7b7-65b2593eb6ab admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:52.461674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b199960-4bfa-41d3-a66e-ca3440886d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:52.462239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:52.462381 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199960-4bfa-41d3-a66e-ca3440886d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:52.462669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199960-4bfa-41d3-a66e-ca3440886d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:52.462888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b199960-4bfa-41d3-a66e-ca3440886d41 admin admin] PATH_INFO: `/v3/projects/fa992541102c467c92cff50ae45186b8/users/b943816026cb4aa49c82f555fc115e2b/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:52.464047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:52.485960 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9d811-490c-4331-b15d-54e48441e01d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:52.493866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:52.494958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:52.495797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:52.496282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] PATH_INFO: `/v3/projects/cae55ca3c2a943259ce886c22848d639` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:52.498074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae55ca3c2a943259ce886c22848d639)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:52.499000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199960-4bfa-41d3-a66e-ca3440886d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa992541102c467c92cff50ae45186b8, user_id=b943816026cb4aa49c82f555fc115e2b, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:52.501076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:52.502187 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b199960-4bfa-41d3-a66e-ca3440886d41 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:52.691792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1031/2080] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:52 2022] PUT /identity/v3/projects/fa992541102c467c92cff50ae45186b8/users/b943816026cb4aa49c82f555fc115e2b/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:52.698989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:52.704245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584092a2-f449-4c53-a345-0b2107abead3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:52.704541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584092a2-f449-4c53-a345-0b2107abead3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:52.704822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584092a2-f449-4c53-a345-0b2107abead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:52.715610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae55ca3c2a943259ce886c22848d639', 'request_id': 'req-ce96be86-f347-4282-a835-5894b5986211'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:52.727330 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae55ca3c2a943259ce886c22848d639', 'request_id': 'req-ce96be86-f347-4282-a835-5894b5986211'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:52.740018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-584092a2-f449-4c53-a345-0b2107abead3 None None] MFA Rules not processed for user `b943816026cb4aa49c82f555fc115e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:52.741278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-584092a2-f449-4c53-a345-0b2107abead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:52.785761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:52.828624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae55ca3c2a943259ce886c22848d639', 'request_id': 'req-ce96be86-f347-4282-a835-5894b5986211'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:52.831893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1032/2081] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:52.846105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584092a2-f449-4c53-a345-0b2107abead3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:52.846457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] The token cache is being invalidate because project cae55ca3c2a943259ce886c22848d639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:52.847407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:52.848604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:52.851174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1050/2082] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:52 2022] DELETE /identity/v3/projects/cae55ca3c2a943259ce886c22848d639 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:52.854498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:52.856100 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:52.917392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584092a2-f449-4c53-a345-0b2107abead3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:52.922566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:52.923179 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:52.923459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:52.923735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:52.924405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:52.926347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:52.929223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96be86-f347-4282-a835-5894b5986211 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:52.934932 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:52.935580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:52.935864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:52.936232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] PATH_INFO: `/v3/users/e298bb28572e422ea59f0939ae0beb9e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:52.937765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e298bb28572e422ea59f0939ae0beb9e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:52.940095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:53.056762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1033/2083] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:52 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:53.061004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:53.063133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:53.130621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6d3e9-19f9-406c-aec2-0df7c9a7f2db admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:53.135545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:53.136158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:53.136472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:53.136755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:53.137400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:53.139334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:53.201455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e298bb28572e422ea59f0939ae0beb9e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:53.202117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e298bb28572e422ea59f0939ae0beb9e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:53.212213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1034/2084] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:53 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:53.216434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e298bb28572e422ea59f0939ae0beb9e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:53.217002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:53.219615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:53.238238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b335d-b8d2-4f36-96a0-1f7fd6499190 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:53.246851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b864ad-71af-4f8a-be61-0b357524a16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:53.247499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b864ad-71af-4f8a-be61-0b357524a16c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:53.247826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b864ad-71af-4f8a-be61-0b357524a16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:53.248150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b864ad-71af-4f8a-be61-0b357524a16c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:53.248933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b864ad-71af-4f8a-be61-0b357524a16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:53.251150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b864ad-71af-4f8a-be61-0b357524a16c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:53.261195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e298bb28572e422ea59f0939ae0beb9e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:53.261814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e298bb28572e422ea59f0939ae0beb9e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:53.261955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1035/2085] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:53.267011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b864ad-71af-4f8a-be61-0b357524a16c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:53.273564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:53.277569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b864ad-71af-4f8a-be61-0b357524a16c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:53.283166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40176bd-024b-4147-96fc-57b943c3921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:53.283785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40176bd-024b-4147-96fc-57b943c3921d admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:53.284062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40176bd-024b-4147-96fc-57b943c3921d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:53.284369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40176bd-024b-4147-96fc-57b943c3921d admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:53.285039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40176bd-024b-4147-96fc-57b943c3921d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:53.287513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40176bd-024b-4147-96fc-57b943c3921d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:53.288151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1051/2086] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:52 2022] DELETE /identity/v3/users/e298bb28572e422ea59f0939ae0beb9e => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:53.300852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1036/2087] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:53.309533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:53.394335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce72ae-bb79-457a-9aec-04e818c091c2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:53.399339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c761d3-1de7-44ac-9d62-91b4adeddd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:53.400048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c761d3-1de7-44ac-9d62-91b4adeddd94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:53.400365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c761d3-1de7-44ac-9d62-91b4adeddd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:53.400681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c761d3-1de7-44ac-9d62-91b4adeddd94 admin admin] PATH_INFO: `/v3/projects/e175c4cc812548db8b60c07d1b9c8f9d/users/d961a71bbc064be09d2b289bbf40aa67/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:53.416456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c761d3-1de7-44ac-9d62-91b4adeddd94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e175c4cc812548db8b60c07d1b9c8f9d, user_id=d961a71bbc064be09d2b289bbf40aa67, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:53.419222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c761d3-1de7-44ac-9d62-91b4adeddd94 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:53.524492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1052/2088] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:53 2022] PUT /identity/v3/projects/e175c4cc812548db8b60c07d1b9c8f9d/users/d961a71bbc064be09d2b289bbf40aa67/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:53.535516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16baad9-a6cf-415a-9fae-70c0f09cfd42 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:53.535802 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16baad9-a6cf-415a-9fae-70c0f09cfd42 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:53.536078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16baad9-a6cf-415a-9fae-70c0f09cfd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:53.570042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c16baad9-a6cf-415a-9fae-70c0f09cfd42 None None] MFA Rules not processed for user `d961a71bbc064be09d2b289bbf40aa67`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:53.573511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c16baad9-a6cf-415a-9fae-70c0f09cfd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:53.607400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:53.670626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1037/2089] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:53.677363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c761d3-1de7-44ac-9d62-91b4adeddd94 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:53.768506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c761d3-1de7-44ac-9d62-91b4adeddd94 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:53.776757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b05e5-8416-448d-a2cf-fd97921427da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:53.777799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b05e5-8416-448d-a2cf-fd97921427da admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:53.778307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b05e5-8416-448d-a2cf-fd97921427da admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:53.778816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b05e5-8416-448d-a2cf-fd97921427da admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:53.779988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b05e5-8416-448d-a2cf-fd97921427da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:53.784474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b05e5-8416-448d-a2cf-fd97921427da admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:53.837762 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16baad9-a6cf-415a-9fae-70c0f09cfd42 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:53.839799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c16baad9-a6cf-415a-9fae-70c0f09cfd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:53.880924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16baad9-a6cf-415a-9fae-70c0f09cfd42 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:53.887082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:53.890733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:53.890733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:53.890733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] PATH_INFO: `/v3/projects/de08a94a63e24f98b8e31609560b33d9` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:53.890733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=de08a94a63e24f98b8e31609560b33d9)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:53.894601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:53.907398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1053/2090] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:53 2022] POST /identity/v3/projects => generated 377 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:53.911237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b05e5-8416-448d-a2cf-fd97921427da admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:54.006659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:54.014558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b05e5-8416-448d-a2cf-fd97921427da admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:54.045642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493df0c3-e8ba-43e6-a72a-6790bb0b7089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:54.255586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493df0c3-e8ba-43e6-a72a-6790bb0b7089 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.255586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493df0c3-e8ba-43e6-a72a-6790bb0b7089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.255586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493df0c3-e8ba-43e6-a72a-6790bb0b7089 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.255586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493df0c3-e8ba-43e6-a72a-6790bb0b7089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:54.255586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493df0c3-e8ba-43e6-a72a-6790bb0b7089 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:54.367832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de08a94a63e24f98b8e31609560b33d9', 'request_id': 'req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:54.377233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de08a94a63e24f98b8e31609560b33d9', 'request_id': 'req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:54.385895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de08a94a63e24f98b8e31609560b33d9', 'request_id': 'req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:54.412438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1054/2091] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:53 2022] POST /identity/v3/users => generated 295 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:54.417252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493df0c3-e8ba-43e6-a72a-6790bb0b7089 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:54.434552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493df0c3-e8ba-43e6-a72a-6790bb0b7089 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:54.499182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] The token cache is being invalidate because project de08a94a63e24f98b8e31609560b33d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:54.500517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:54.501968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d506dfed-70fb-47c3-90ad-8e2b0454f1ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:54.504698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1038/2092] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:53 2022] DELETE /identity/v3/projects/de08a94a63e24f98b8e31609560b33d9 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:54.510073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5230f-c769-4b3d-ab0b-1348e8d9b3b6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.510362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5230f-c769-4b3d-ab0b-1348e8d9b3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.511514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5230f-c769-4b3d-ab0b-1348e8d9b3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.536029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4ad5230f-c769-4b3d-ab0b-1348e8d9b3b6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:54.538320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4ad5230f-c769-4b3d-ab0b-1348e8d9b3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:54.539905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:54.540646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.540996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.541362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.542114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:54.544439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:54.562713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1055/2093] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:54.564930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:54.566885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:54.583853 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744060aa-938b-4d22-824f-f29a6db3edc4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:54.586658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1039/2094] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:54.593528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ccbc89-5b70-449d-9582-a4285ec75e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:54.594401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccbc89-5b70-449d-9582-a4285ec75e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.594793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccbc89-5b70-449d-9582-a4285ec75e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.595174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccbc89-5b70-449d-9582-a4285ec75e0c admin admin] PATH_INFO: `/v3/projects/42ce13e2223a4a3297b4eb0892afaefe/users/d8b6313ca6f6402b80783b2622ea41c6/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.596779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813ae85-8a59-4efa-afcf-b70a5bdd81c8 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.597287 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813ae85-8a59-4efa-afcf-b70a5bdd81c8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.597788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813ae85-8a59-4efa-afcf-b70a5bdd81c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.614209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ccbc89-5b70-449d-9582-a4285ec75e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ce13e2223a4a3297b4eb0892afaefe, user_id=d8b6313ca6f6402b80783b2622ea41c6, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:54.617385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ccbc89-5b70-449d-9582-a4285ec75e0c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:54.621844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-0813ae85-8a59-4efa-afcf-b70a5bdd81c8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:54.624759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0813ae85-8a59-4efa-afcf-b70a5bdd81c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:54.637831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1040/2095] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:54.638976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813ae85-8a59-4efa-afcf-b70a5bdd81c8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:54.652790 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813ae85-8a59-4efa-afcf-b70a5bdd81c8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:54.668514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:54.671139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.671618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.672299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.676496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:54.678417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:54.678544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:54.706475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1041/2096] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:54.713505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:54.744338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1056/2097] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:54 2022] PUT /identity/v3/projects/42ce13e2223a4a3297b4eb0892afaefe/users/d8b6313ca6f6402b80783b2622ea41c6/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:54.748883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ccbc89-5b70-449d-9582-a4285ec75e0c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:54.763706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1fc46-905d-4ebd-be2e-d20c0675bc7b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:54.846966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ccbc89-5b70-449d-9582-a4285ec75e0c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:54.852763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12862ffc-ed76-4426-a086-30dfbd5060b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:54.854808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862ffc-ed76-4426-a086-30dfbd5060b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.854808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862ffc-ed76-4426-a086-30dfbd5060b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.855235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12862ffc-ed76-4426-a086-30dfbd5060b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.855928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12862ffc-ed76-4426-a086-30dfbd5060b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:54.857598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12862ffc-ed76-4426-a086-30dfbd5060b4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:54.867113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0cb01-d018-4ab0-b0e7-d85d74f23bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:54.867736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0cb01-d018-4ab0-b0e7-d85d74f23bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.868076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0cb01-d018-4ab0-b0e7-d85d74f23bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.868437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0cb01-d018-4ab0-b0e7-d85d74f23bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.869221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0cb01-d018-4ab0-b0e7-d85d74f23bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:54.871162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0cb01-d018-4ab0-b0e7-d85d74f23bbd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:54.872667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1057/2098] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:54.878051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12862ffc-ed76-4426-a086-30dfbd5060b4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:54.889529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12862ffc-ed76-4426-a086-30dfbd5060b4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:54.899551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94122cae-59fe-46bb-93ee-5065beb8f31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:54.900765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122cae-59fe-46bb-93ee-5065beb8f31f admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.901279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122cae-59fe-46bb-93ee-5065beb8f31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.901796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94122cae-59fe-46bb-93ee-5065beb8f31f admin admin] PATH_INFO: `/v3/projects/42ce13e2223a4a3297b4eb0892afaefe/users/d8b6313ca6f6402b80783b2622ea41c6/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.906377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94122cae-59fe-46bb-93ee-5065beb8f31f admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ce13e2223a4a3297b4eb0892afaefe, user_id=d8b6313ca6f6402b80783b2622ea41c6, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:54.911464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94122cae-59fe-46bb-93ee-5065beb8f31f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:54.924918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1058/2099] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:54 2022] PUT /identity/v3/projects/42ce13e2223a4a3297b4eb0892afaefe/users/d8b6313ca6f6402b80783b2622ea41c6/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:54.938487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a02b-2ad6-4ab8-97ce-a8ed673c5879 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:54.939459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a02b-2ad6-4ab8-97ce-a8ed673c5879 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:54.939762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a02b-2ad6-4ab8-97ce-a8ed673c5879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:54.969657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:54.980622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-27f2a02b-2ad6-4ab8-97ce-a8ed673c5879 None None] MFA Rules not processed for user `d8b6313ca6f6402b80783b2622ea41c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:54.986965 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1042/2100] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:54 2022] POST /identity/v3/projects => generated 379 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:54.987275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-27f2a02b-2ad6-4ab8-97ce-a8ed673c5879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:54.989091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0cb01-d018-4ab0-b0e7-d85d74f23bbd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:55.130389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1059/2101] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:55.135426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0cb01-d018-4ab0-b0e7-d85d74f23bbd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:55.143215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a02b-2ad6-4ab8-97ce-a8ed673c5879 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:55.144713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-27f2a02b-2ad6-4ab8-97ce-a8ed673c5879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:55.145530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84058123-0e15-4cb9-8d05-aebda08b17c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:55.146670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84058123-0e15-4cb9-8d05-aebda08b17c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:55.147331 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84058123-0e15-4cb9-8d05-aebda08b17c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:55.147930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84058123-0e15-4cb9-8d05-aebda08b17c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:55.149173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84058123-0e15-4cb9-8d05-aebda08b17c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:55.153274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84058123-0e15-4cb9-8d05-aebda08b17c5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:55.213004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a02b-2ad6-4ab8-97ce-a8ed673c5879 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:55.217566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:55.221491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:55.221491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:55.221491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:55.221491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:55.291710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1043/2102] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:54 2022] POST /identity/v3/users => generated 290 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:55.296553 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84058123-0e15-4cb9-8d05-aebda08b17c5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:55.299206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:55.301576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:55.312611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84058123-0e15-4cb9-8d05-aebda08b17c5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:55.317062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1060/2103] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:39:55.320158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e190c5-356f-4e88-b295-396d7b6d74da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:55.324832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e190c5-356f-4e88-b295-396d7b6d74da admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:55.324832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e190c5-356f-4e88-b295-396d7b6d74da admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:55.324832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e190c5-356f-4e88-b295-396d7b6d74da admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:55.324832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e190c5-356f-4e88-b295-396d7b6d74da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:55.324832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e190c5-356f-4e88-b295-396d7b6d74da admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:55.343099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1044/2104] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:55.348546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:55.375065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4acc1-7abf-4a31-9349-92ee6b57ac5f service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:55.392807 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5744b-370f-404d-a161-405d44ac13bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:55.401996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5744b-370f-404d-a161-405d44ac13bf admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:55.401996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5744b-370f-404d-a161-405d44ac13bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:55.401996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5744b-370f-404d-a161-405d44ac13bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:55.401996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5744b-370f-404d-a161-405d44ac13bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:55.404904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5744b-370f-404d-a161-405d44ac13bf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:55.438658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1061/2105] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:55.443205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e190c5-356f-4e88-b295-396d7b6d74da admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:55.457158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e190c5-356f-4e88-b295-396d7b6d74da admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:55.472482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:55.474384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:55.475361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:55.476208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] PATH_INFO: `/v3/projects/a26934f04fae43ffbad78bcf9108f8cf/users/a4a4f64021854a36ad52ee1e25c0306a/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:55.504331 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26934f04fae43ffbad78bcf9108f8cf, user_id=a4a4f64021854a36ad52ee1e25c0306a, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:55.507577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:55.564352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:55.571843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1045/2106] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:55 2022] PUT /identity/v3/projects/a26934f04fae43ffbad78bcf9108f8cf/users/a4a4f64021854a36ad52ee1e25c0306a/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:55.584511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17105-2608-449f-9c6e-5c3f53ab1d08 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:55.584877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17105-2608-449f-9c6e-5c3f53ab1d08 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:55.585352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be17105-2608-449f-9c6e-5c3f53ab1d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:55.620276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0be17105-2608-449f-9c6e-5c3f53ab1d08 None None] MFA Rules not processed for user `a4a4f64021854a36ad52ee1e25c0306a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:55.621579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0be17105-2608-449f-9c6e-5c3f53ab1d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:55.700492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:55.702819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:55.709735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1062/2107] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:55.714256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be17105-2608-449f-9c6e-5c3f53ab1d08 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:55.786027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be17105-2608-449f-9c6e-5c3f53ab1d08 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:55.791343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92205762-a18d-4dbf-96e8-fa4ca06bd7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:55.791968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92205762-a18d-4dbf-96e8-fa4ca06bd7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:55.792367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92205762-a18d-4dbf-96e8-fa4ca06bd7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:55.792710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92205762-a18d-4dbf-96e8-fa4ca06bd7c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:55.793363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92205762-a18d-4dbf-96e8-fa4ca06bd7c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:55.797389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f1c1d9-43b9-41d1-b649-9ec661839164 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:55.798343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92205762-a18d-4dbf-96e8-fa4ca06bd7c4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:55.807031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:55.808278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:55.810251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:55.810251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] PATH_INFO: `/v3/users/24eba350ada540c78e27556519d25ce1` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:55.811622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24eba350ada540c78e27556519d25ce1)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:55.814603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:56.021068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1063/2108] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:55 2022] POST /identity/v3/projects => generated 379 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:56.022755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92205762-a18d-4dbf-96e8-fa4ca06bd7c4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:56.116733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92205762-a18d-4dbf-96e8-fa4ca06bd7c4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:56.130797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:56.134268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:56.134268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:56.134268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:56.134268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:56.137012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:56.145634 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24eba350ada540c78e27556519d25ce1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.146948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24eba350ada540c78e27556519d25ce1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.161645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24eba350ada540c78e27556519d25ce1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.162845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24eba350ada540c78e27556519d25ce1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.276081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:56.283761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b61f72b4-b1b4-4f65-87f3-9f0974b58125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24eba350ada540c78e27556519d25ce1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.286717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1064/2109] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:56 2022] POST /identity/v3/users => generated 290 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:56.291545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:56.299936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:56.319686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029753-4e7a-4ed0-a964-e32d8004c808 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:56.322209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1046/2110] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:55 2022] DELETE /identity/v3/users/24eba350ada540c78e27556519d25ce1 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:56.336626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cf175-74ae-4e5c-a286-7240f5680db8 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:56.336626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cf175-74ae-4e5c-a286-7240f5680db8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:56.336626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cf175-74ae-4e5c-a286-7240f5680db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:56.405724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9a9cf175-74ae-4e5c-a286-7240f5680db8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:56.405913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a9cf175-74ae-4e5c-a286-7240f5680db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:56.495020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c031902-dedd-4781-98db-8076179e37a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:56.496380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c031902-dedd-4781-98db-8076179e37a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:56.496848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c031902-dedd-4781-98db-8076179e37a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:56.497268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c031902-dedd-4781-98db-8076179e37a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:56.498118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c031902-dedd-4781-98db-8076179e37a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:56.510504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c031902-dedd-4781-98db-8076179e37a1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:56.515842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:56.523043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1047/2111] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:56.524552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cf175-74ae-4e5c-a286-7240f5680db8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:56.536340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1065/2112] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:56.539323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9a9cf175-74ae-4e5c-a286-7240f5680db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:56.556250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c031902-dedd-4781-98db-8076179e37a1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:56.559324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c031902-dedd-4781-98db-8076179e37a1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:56.581541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:56.582200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:56.600460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:56.600460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:56.600460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:56.600460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:56.600928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cf175-74ae-4e5c-a286-7240f5680db8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:56.600928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:56.601280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:56.601475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:56.602398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] PATH_INFO: `/v3/users/81759b101bc44b8d99638d5735e9818f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:56.604986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81759b101bc44b8d99638d5735e9818f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:56.608198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:56.612611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1066/2113] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:56.614270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:56.621144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:56.727776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034645ad-00ad-4939-93ff-72472c62529c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:56.736977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717b8e-b0fb-444f-82f1-a988cbb48f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:56.738039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717b8e-b0fb-444f-82f1-a988cbb48f88 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:56.738870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717b8e-b0fb-444f-82f1-a988cbb48f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:56.739148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717b8e-b0fb-444f-82f1-a988cbb48f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:56.763418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717b8e-b0fb-444f-82f1-a988cbb48f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:56.764988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717b8e-b0fb-444f-82f1-a988cbb48f88 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:56.782821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1067/2114] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:56.782821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717b8e-b0fb-444f-82f1-a988cbb48f88 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:56.792941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717b8e-b0fb-444f-82f1-a988cbb48f88 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:56.799889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2f5b9-b0be-456a-a343-bbe5d85f993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:56.800165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f5b9-b0be-456a-a343-bbe5d85f993f admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:56.800313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f5b9-b0be-456a-a343-bbe5d85f993f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:56.800577 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2f5b9-b0be-456a-a343-bbe5d85f993f admin admin] PATH_INFO: `/v3/projects/d856b049b2244fb7a87e5179f3d45cd6/users/3cf8ac51fa3c45e195c94f498148b18a/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:56.824391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2f5b9-b0be-456a-a343-bbe5d85f993f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d856b049b2244fb7a87e5179f3d45cd6, user_id=3cf8ac51fa3c45e195c94f498148b18a, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:56.828226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2f5b9-b0be-456a-a343-bbe5d85f993f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:56.908955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81759b101bc44b8d99638d5735e9818f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.910338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81759b101bc44b8d99638d5735e9818f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.924801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81759b101bc44b8d99638d5735e9818f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.949484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81759b101bc44b8d99638d5735e9818f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:56.977244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1068/2115] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:56 2022] PUT /identity/v3/projects/d856b049b2244fb7a87e5179f3d45cd6/users/3cf8ac51fa3c45e195c94f498148b18a/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:56.991973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2677f34-607f-413a-8e85-9b377d18cf02 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:56.992314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2677f34-607f-413a-8e85-9b377d18cf02 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:56.992591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2677f34-607f-413a-8e85-9b377d18cf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:57.026324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:57.030952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f2677f34-607f-413a-8e85-9b377d18cf02 None None] MFA Rules not processed for user `3cf8ac51fa3c45e195c94f498148b18a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:57.030952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2677f34-607f-413a-8e85-9b377d18cf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:57.084150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0bb33a30-4450-42b8-9ec7-b1c967b92c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81759b101bc44b8d99638d5735e9818f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:57.115182 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1048/2116] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:39:56 2022] DELETE /identity/v3/users/81759b101bc44b8d99638d5735e9818f => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:57.121149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd04b30-68e1-4b40-9cbc-08a975ec4389 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:57.121567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd04b30-68e1-4b40-9cbc-08a975ec4389 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:57.124673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd04b30-68e1-4b40-9cbc-08a975ec4389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:57.133129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1069/2117] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:57.138558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2677f34-607f-413a-8e85-9b377d18cf02 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:57.141757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f2677f34-607f-413a-8e85-9b377d18cf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:57.150211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fbd04b30-68e1-4b40-9cbc-08a975ec4389 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:57.152355 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fbd04b30-68e1-4b40-9cbc-08a975ec4389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:57.228637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2677f34-607f-413a-8e85-9b377d18cf02 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:57.283304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1049/2118] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:57.348780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd04b30-68e1-4b40-9cbc-08a975ec4389 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:57.351098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67be1c-5d80-4d2c-ae3e-bac1bd33f76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:57.351751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67be1c-5d80-4d2c-ae3e-bac1bd33f76c admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:57.352035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67be1c-5d80-4d2c-ae3e-bac1bd33f76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:57.352374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67be1c-5d80-4d2c-ae3e-bac1bd33f76c admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:57.353043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67be1c-5d80-4d2c-ae3e-bac1bd33f76c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:57.354883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fbd04b30-68e1-4b40-9cbc-08a975ec4389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:57.355257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67be1c-5d80-4d2c-ae3e-bac1bd33f76c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:57.522920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1070/2119] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:57 2022] POST /identity/v3/projects => generated 379 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:57.523833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd04b30-68e1-4b40-9cbc-08a975ec4389 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:57.524456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67be1c-5d80-4d2c-ae3e-bac1bd33f76c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:57.532850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:57.665910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67be1c-5d80-4d2c-ae3e-bac1bd33f76c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:57.673078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9b11-e6ef-4a01-9258-28deb5519cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:57.673737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9b11-e6ef-4a01-9258-28deb5519cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:57.674027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9b11-e6ef-4a01-9258-28deb5519cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:57.674387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9b11-e6ef-4a01-9258-28deb5519cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:57.675283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc9b11-e6ef-4a01-9258-28deb5519cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:57.678997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc9b11-e6ef-4a01-9258-28deb5519cd6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:57.720859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864551-bfb5-44cd-9c63-c4a880d0c325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:57.722885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864551-bfb5-44cd-9c63-c4a880d0c325 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:57.722885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864551-bfb5-44cd-9c63-c4a880d0c325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:57.722885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864551-bfb5-44cd-9c63-c4a880d0c325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:57.732501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07864551-bfb5-44cd-9c63-c4a880d0c325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:57.734812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07864551-bfb5-44cd-9c63-c4a880d0c325 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:57.747599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1050/2120] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:57.851524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1071/2121] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:57 2022] POST /identity/v3/users => generated 296 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:57.859798 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864551-bfb5-44cd-9c63-c4a880d0c325 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:57.908223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864551-bfb5-44cd-9c63-c4a880d0c325 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:57.925498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:57.926527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:57.927026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:57.927755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:57.929088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:57.933115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:57.960312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1051/2122] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:57.965920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9b11-e6ef-4a01-9258-28deb5519cd6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:58.004268 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9b11-e6ef-4a01-9258-28deb5519cd6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:58.013989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:58.014859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:58.017259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:58.043838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5264e-280d-4726-837b-25f2f4eedf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:58.044373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5264e-280d-4726-837b-25f2f4eedf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:58.044627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5264e-280d-4726-837b-25f2f4eedf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:58.044902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5264e-280d-4726-837b-25f2f4eedf9a admin admin] PATH_INFO: `/v3/projects/87e79ae44b7041f8ae136316943f0b1e/users/605b5229718d44c5bd25e52d1af28264/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:58.093069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5264e-280d-4726-837b-25f2f4eedf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e79ae44b7041f8ae136316943f0b1e, user_id=605b5229718d44c5bd25e52d1af28264, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:58.110538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5264e-280d-4726-837b-25f2f4eedf9a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:58.144796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db3b61-d686-4d76-83ba-c40dce24cfaa admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:58.149031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:58.149650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:58.149934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:58.150213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:58.152144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:58.235012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:58.235012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:58.242935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1052/2123] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:58.244121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:58.246350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:58.272113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1072/2124] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:57 2022] PUT /identity/v3/projects/87e79ae44b7041f8ae136316943f0b1e/users/605b5229718d44c5bd25e52d1af28264/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:58.274393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd1cc5-fda9-4127-862e-47144dc9d432 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:58.277501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5264e-280d-4726-837b-25f2f4eedf9a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:58.458687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:58.459386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:58.459751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:58.460040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] PATH_INFO: `/v3/projects/e996ffc67446441f9a247f4c63ff1885` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:58.461963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e996ffc67446441f9a247f4c63ff1885)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:58.464469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:58.510838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5264e-280d-4726-837b-25f2f4eedf9a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:58.522951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ba9c-2bfd-47c7-967a-f2b538a032a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:58.523911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ba9c-2bfd-47c7-967a-f2b538a032a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:58.525395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ba9c-2bfd-47c7-967a-f2b538a032a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:58.525395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81ba9c-2bfd-47c7-967a-f2b538a032a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:58.525610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ba9c-2bfd-47c7-967a-f2b538a032a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:58.527498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81ba9c-2bfd-47c7-967a-f2b538a032a0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:58.540642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1073/2125] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:39:58.545011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ba9c-2bfd-47c7-967a-f2b538a032a0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:58.573456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81ba9c-2bfd-47c7-967a-f2b538a032a0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:58.581382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a0f06-803f-480b-b7a7-5b9d75e70993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:58.582656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a0f06-803f-480b-b7a7-5b9d75e70993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:58.583245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a0f06-803f-480b-b7a7-5b9d75e70993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:58.583633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a0f06-803f-480b-b7a7-5b9d75e70993 admin admin] PATH_INFO: `/v3/projects/87e79ae44b7041f8ae136316943f0b1e/users/605b5229718d44c5bd25e52d1af28264/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:58.586636 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a0f06-803f-480b-b7a7-5b9d75e70993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87e79ae44b7041f8ae136316943f0b1e, user_id=605b5229718d44c5bd25e52d1af28264, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:58.590398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a0f06-803f-480b-b7a7-5b9d75e70993 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:58.617358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1074/2126] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:39:58 2022] PUT /identity/v3/projects/87e79ae44b7041f8ae136316943f0b1e/users/605b5229718d44c5bd25e52d1af28264/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:39:58.619687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b845dd-338f-490b-bc44-a75f9042e473 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:58.620076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b845dd-338f-490b-bc44-a75f9042e473 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:58.620464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b845dd-338f-490b-bc44-a75f9042e473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:58.656561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-00b845dd-338f-490b-bc44-a75f9042e473 None None] MFA Rules not processed for user `605b5229718d44c5bd25e52d1af28264`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:58.658320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-00b845dd-338f-490b-bc44-a75f9042e473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:58.711670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e996ffc67446441f9a247f4c63ff1885', 'request_id': 'req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:58.729239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e996ffc67446441f9a247f4c63ff1885', 'request_id': 'req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:58.740912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e996ffc67446441f9a247f4c63ff1885', 'request_id': 'req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:58.766645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1075/2127] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:58.767681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b845dd-338f-490b-bc44-a75f9042e473 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:58.769628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-00b845dd-338f-490b-bc44-a75f9042e473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:58.868297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b845dd-338f-490b-bc44-a75f9042e473 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:58.886124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:58.890628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] The token cache is being invalidate because project e996ffc67446441f9a247f4c63ff1885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:58.894370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:58.894370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:58.899172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1053/2128] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:58 2022] DELETE /identity/v3/projects/e996ffc67446441f9a247f4c63ff1885 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:58.899172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:58.900707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:58.956961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:58.957930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:58.958172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:58.958451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] PATH_INFO: `/v3/projects/4e435478690c4063a3446055972ca54c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:58.959715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e435478690c4063a3446055972ca54c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:58.961781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:58.983022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442fd3c-4600-4d51-8bc4-3572cf54f6d5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:58.990290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c810b-2af2-49ce-9ab1-35da46243e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:58.994039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c810b-2af2-49ce-9ab1-35da46243e59 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:58.994039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c810b-2af2-49ce-9ab1-35da46243e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:58.994039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c810b-2af2-49ce-9ab1-35da46243e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:58.995436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0e9c810b-2af2-49ce-9ab1-35da46243e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:59.043633 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:59.088235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c810b-2af2-49ce-9ab1-35da46243e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:59.090652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c810b-2af2-49ce-9ab1-35da46243e59 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:59.103693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1054/2129] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:59.109075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9bd79-957e-44ec-90dd-275375861589 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.110016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9bd79-957e-44ec-90dd-275375861589 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.110670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9bd79-957e-44ec-90dd-275375861589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.131435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e435478690c4063a3446055972ca54c', 'request_id': 'req-41c13e58-8509-4127-91de-cf6559c1ebee'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:59.141940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-afe9bd79-957e-44ec-90dd-275375861589 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:59.143333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-afe9bd79-957e-44ec-90dd-275375861589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:59.145736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e435478690c4063a3446055972ca54c', 'request_id': 'req-41c13e58-8509-4127-91de-cf6559c1ebee'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:59.228718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1055/2130] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:59.238621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa79103-d536-4c96-aa61-b0e0cb641644 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.239057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa79103-d536-4c96-aa61-b0e0cb641644 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.239341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa79103-d536-4c96-aa61-b0e0cb641644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.251217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e435478690c4063a3446055972ca54c', 'request_id': 'req-41c13e58-8509-4127-91de-cf6559c1ebee'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:59.274197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] The token cache is being invalidate because project 4e435478690c4063a3446055972ca54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:39:59.274594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:59.275394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:39:59.277909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1076/2131] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:39:58 2022] DELETE /identity/v3/projects/4e435478690c4063a3446055972ca54c => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:39:59.279199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.280869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:59.282085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2fa79103-d536-4c96-aa61-b0e0cb641644 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:59.285131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2fa79103-d536-4c96-aa61-b0e0cb641644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:59.294701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1056/2132] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:39:59.296586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed77a-1d06-4850-9541-dbd2f93d56d6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.297041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed77a-1d06-4850-9541-dbd2f93d56d6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.297761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed77a-1d06-4850-9541-dbd2f93d56d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.321541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ce9ed77a-1d06-4850-9541-dbd2f93d56d6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:59.323044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ce9ed77a-1d06-4850-9541-dbd2f93d56d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:59.334099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1057/2133] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:59.335095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ed77a-1d06-4850-9541-dbd2f93d56d6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.347017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ed77a-1d06-4850-9541-dbd2f93d56d6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:59.353935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52630e3a-408b-43f8-a458-6dbf21bcc9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:59.355006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52630e3a-408b-43f8-a458-6dbf21bcc9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.355006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52630e3a-408b-43f8-a458-6dbf21bcc9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.355268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52630e3a-408b-43f8-a458-6dbf21bcc9e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.355996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52630e3a-408b-43f8-a458-6dbf21bcc9e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:59.356441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52630e3a-408b-43f8-a458-6dbf21bcc9e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:59.358208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52630e3a-408b-43f8-a458-6dbf21bcc9e1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:59.371874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1058/2134] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:59.373570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04741a66-8b08-4f8e-b36f-49ea7b188979 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.374509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04741a66-8b08-4f8e-b36f-49ea7b188979 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.374943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04741a66-8b08-4f8e-b36f-49ea7b188979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.385810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c13e58-8509-4127-91de-cf6559c1ebee admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:59.399661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:59.400310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.400595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.400864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.401845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-04741a66-8b08-4f8e-b36f-49ea7b188979 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:39:59.403141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:59.403278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-04741a66-8b08-4f8e-b36f-49ea7b188979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:39:59.416343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1059/2135] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:39:59.417225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04741a66-8b08-4f8e-b36f-49ea7b188979 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.450966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04741a66-8b08-4f8e-b36f-49ea7b188979 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:59.466439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5de6b-f83d-464e-8ab2-32c7317a8d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:59.467397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5de6b-f83d-464e-8ab2-32c7317a8d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.468781 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5de6b-f83d-464e-8ab2-32c7317a8d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.469267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5de6b-f83d-464e-8ab2-32c7317a8d1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.470007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5de6b-f83d-464e-8ab2-32c7317a8d1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:59.472791 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5de6b-f83d-464e-8ab2-32c7317a8d1e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:59.502534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:59.530865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:59.534111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:59.550095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1077/2136] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:39:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:39:59.563428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.585834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3dc86-bc91-491b-86fd-b9c710e00ede service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:59.593283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:59.602159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1060/2137] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:59 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:59.602272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.602272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.602272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.602751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:39:59.603026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:59.605167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5de6b-f83d-464e-8ab2-32c7317a8d1e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.607837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:59.630488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1078/2138] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:39:59.634769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.651873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ab256-a3d1-437e-82d6-20946419c32e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:59.709801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5de6b-f83d-464e-8ab2-32c7317a8d1e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:59.715850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bef7b5-94da-4e2a-939b-7f8ce5e34a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:59.722246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bef7b5-94da-4e2a-939b-7f8ce5e34a43 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.722246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bef7b5-94da-4e2a-939b-7f8ce5e34a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.722246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bef7b5-94da-4e2a-939b-7f8ce5e34a43 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.722246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bef7b5-94da-4e2a-939b-7f8ce5e34a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:59.722246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bef7b5-94da-4e2a-939b-7f8ce5e34a43 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:59.735285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cf306-6698-40a2-8340-738e31b5c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:59.736463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cf306-6698-40a2-8340-738e31b5c1ac admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.736984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cf306-6698-40a2-8340-738e31b5c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.739130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cf306-6698-40a2-8340-738e31b5c1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.740578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cf306-6698-40a2-8340-738e31b5c1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:59.751082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cf306-6698-40a2-8340-738e31b5c1ac admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:59.777034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:39:59.847876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1061/2139] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:59 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:39:59.849768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bef7b5-94da-4e2a-939b-7f8ce5e34a43 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.862609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bef7b5-94da-4e2a-939b-7f8ce5e34a43 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:39:59.873787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5d9f-da68-4c4b-aed8-9a785d5184dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:39:59.878135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5d9f-da68-4c4b-aed8-9a785d5184dc admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:39:59.878410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5d9f-da68-4c4b-aed8-9a785d5184dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:39:59.882746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1079/2140] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:39:59 2022] POST /identity/v3/projects => generated 375 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:39:59.883039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5d9f-da68-4c4b-aed8-9a785d5184dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:39:59.883763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a5d9f-da68-4c4b-aed8-9a785d5184dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:39:59.887688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cf306-6698-40a2-8340-738e31b5c1ac admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.890826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a5d9f-da68-4c4b-aed8-9a785d5184dc admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:39:59.910047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1062/2141] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:39:59.914866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5d9f-da68-4c4b-aed8-9a785d5184dc admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:39:59.929205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5d9f-da68-4c4b-aed8-9a785d5184dc admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:00.020669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8311f3-21a8-483b-8953-3dc77f3bc694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:00.021537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8311f3-21a8-483b-8953-3dc77f3bc694 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.021816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8311f3-21a8-483b-8953-3dc77f3bc694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.022090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8311f3-21a8-483b-8953-3dc77f3bc694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.022760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8311f3-21a8-483b-8953-3dc77f3bc694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:00.025944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8311f3-21a8-483b-8953-3dc77f3bc694 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:00.027007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cf306-6698-40a2-8340-738e31b5c1ac admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:00.036699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7c0f8-fccc-4da9-a610-8013b89eb3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:00.037617 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7c0f8-fccc-4da9-a610-8013b89eb3fd admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.037904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7c0f8-fccc-4da9-a610-8013b89eb3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.038177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7c0f8-fccc-4da9-a610-8013b89eb3fd admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.041687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1063/2142] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:00.042031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7c0f8-fccc-4da9-a610-8013b89eb3fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:00.042031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7c0f8-fccc-4da9-a610-8013b89eb3fd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:00.043335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8311f3-21a8-483b-8953-3dc77f3bc694 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:00.058124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8311f3-21a8-483b-8953-3dc77f3bc694 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:00.066827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99952fe8-6b66-4fe5-8418-b7842f48490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:00.067805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99952fe8-6b66-4fe5-8418-b7842f48490c admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.068347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99952fe8-6b66-4fe5-8418-b7842f48490c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.070457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99952fe8-6b66-4fe5-8418-b7842f48490c admin admin] PATH_INFO: `/v3/projects/70d0978c718d4704905e96d4ef197796/users/20c258b3ad7340acbe00bc4c790880a0/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.095447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99952fe8-6b66-4fe5-8418-b7842f48490c admin admin] RBAC: Authorizing `identity:create_grant(project_id=70d0978c718d4704905e96d4ef197796, user_id=20c258b3ad7340acbe00bc4c790880a0, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:00.098314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99952fe8-6b66-4fe5-8418-b7842f48490c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:00.201733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1080/2143] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:39:59 2022] POST /identity/v3/users => generated 288 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:40:00.203916 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1064/2144] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:00 2022] PUT /identity/v3/projects/70d0978c718d4704905e96d4ef197796/users/20c258b3ad7340acbe00bc4c790880a0/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:00.209915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7c0f8-fccc-4da9-a610-8013b89eb3fd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:00.215947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f594c-6662-49ee-9b11-d1a808c2247c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.216364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f594c-6662-49ee-9b11-d1a808c2247c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.217192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f594c-6662-49ee-9b11-d1a808c2247c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.217540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7c0f8-fccc-4da9-a610-8013b89eb3fd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:00.255319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ed0f594c-6662-49ee-9b11-d1a808c2247c None None] MFA Rules not processed for user `20c258b3ad7340acbe00bc4c790880a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:00.256960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ed0f594c-6662-49ee-9b11-d1a808c2247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:00.319910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ad4c2-78e8-40a5-8ebf-141dafdd6c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:00.319910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad4c2-78e8-40a5-8ebf-141dafdd6c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.320373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad4c2-78e8-40a5-8ebf-141dafdd6c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.320554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad4c2-78e8-40a5-8ebf-141dafdd6c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.321958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ad4c2-78e8-40a5-8ebf-141dafdd6c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:00.325313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ad4c2-78e8-40a5-8ebf-141dafdd6c3f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:00.348864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1081/2145] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:00.351324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ad4c2-78e8-40a5-8ebf-141dafdd6c3f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:00.365637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ad4c2-78e8-40a5-8ebf-141dafdd6c3f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:00.377005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:00.377690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.377974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.378272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.380862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:00.391085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:00.397853 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1065/2146] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:40:00.412977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1082/2147] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:00.415312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f594c-6662-49ee-9b11-d1a808c2247c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:00.437180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f594c-6662-49ee-9b11-d1a808c2247c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:00.442045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:00.444053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe16ef3-89d5-4b1b-a7e7-f2e03fecef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:00.444500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:00.444986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe16ef3-89d5-4b1b-a7e7-f2e03fecef7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.445370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe16ef3-89d5-4b1b-a7e7-f2e03fecef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.445758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe16ef3-89d5-4b1b-a7e7-f2e03fecef7b admin admin] PATH_INFO: `/v3/projects/79f8a4e5bb274136b40fdfce9300be78/users/2475da9c30184832bb205aac12ec6840/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.477409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe16ef3-89d5-4b1b-a7e7-f2e03fecef7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f8a4e5bb274136b40fdfce9300be78, user_id=2475da9c30184832bb205aac12ec6840, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:00.480371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe16ef3-89d5-4b1b-a7e7-f2e03fecef7b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:00.526091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3a920-006b-41a7-9c56-e66dda57bd8d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:00.530997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f98f52-301b-48ec-abd6-7b06d72ff0f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:00.531943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f98f52-301b-48ec-abd6-7b06d72ff0f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.532270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f98f52-301b-48ec-abd6-7b06d72ff0f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.532656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f98f52-301b-48ec-abd6-7b06d72ff0f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.543495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f98f52-301b-48ec-abd6-7b06d72ff0f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:00.545003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f98f52-301b-48ec-abd6-7b06d72ff0f0 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:00.573707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1083/2148] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:00.597669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1066/2149] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:00 2022] PUT /identity/v3/projects/79f8a4e5bb274136b40fdfce9300be78/users/2475da9c30184832bb205aac12ec6840/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:00.617645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb942b-1624-452d-8500-094407f225ef None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:00.618016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb942b-1624-452d-8500-094407f225ef None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:00.618377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb942b-1624-452d-8500-094407f225ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:00.682874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3ceb942b-1624-452d-8500-094407f225ef None None] MFA Rules not processed for user `2475da9c30184832bb205aac12ec6840`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:00.682874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3ceb942b-1624-452d-8500-094407f225ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:00.769014 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1084/2150] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:00.853860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:00.937394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe16ef3-89d5-4b1b-a7e7-f2e03fecef7b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:01.032034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe16ef3-89d5-4b1b-a7e7-f2e03fecef7b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:01.036452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:01.037212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:01.037615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:01.037968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:01.073269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:01.073269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:01.073652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1067/2151] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:01.370254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:01.791372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb942b-1624-452d-8500-094407f225ef None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:01.804264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb942b-1624-452d-8500-094407f225ef None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:01.811679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:01.813114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:01.813669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:01.814158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:01.817523 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:01.914041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:01.915851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:01.927929 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1085/2152] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:02.238605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:02.240142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:02.347581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dfe3c-29b2-4aa9-859c-cf205ba948f4 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:02.356955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:02.357687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:02.358040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:02.359135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] PATH_INFO: `/v3/users/0108005591b14c5081d460f5c1a7526f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:02.360862 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0108005591b14c5081d460f5c1a7526f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:02.362936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:02.794584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0108005591b14c5081d460f5c1a7526f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:02.795166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0108005591b14c5081d460f5c1a7526f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:02.803099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0108005591b14c5081d460f5c1a7526f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:02.803834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0108005591b14c5081d460f5c1a7526f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:02.970157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b87a8221-f3dd-4ada-be96-e94d11223c91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0108005591b14c5081d460f5c1a7526f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:02.991958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1068/2153] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:02 2022] DELETE /identity/v3/users/0108005591b14c5081d460f5c1a7526f => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:03.295353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:03.309306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0c67e-e18a-4b39-b98e-951f99cabfe4 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:03.390985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:03.391674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:03.391954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:03.392278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] PATH_INFO: `/v3/projects/77b5eebd2706440badffd858826d5c79` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:03.393622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b5eebd2706440badffd858826d5c79)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:03.395544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:03.594437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b5eebd2706440badffd858826d5c79', 'request_id': 'req-b2220725-9c80-43ae-8120-f016610f9ea6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:03.604476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b5eebd2706440badffd858826d5c79', 'request_id': 'req-b2220725-9c80-43ae-8120-f016610f9ea6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:03.693368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b5eebd2706440badffd858826d5c79', 'request_id': 'req-b2220725-9c80-43ae-8120-f016610f9ea6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:03.712118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] The token cache is being invalidate because project 77b5eebd2706440badffd858826d5c79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:03.712118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:03.712118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b2220725-9c80-43ae-8120-f016610f9ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:03.713621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1086/2154] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:03 2022] DELETE /identity/v3/projects/77b5eebd2706440badffd858826d5c79 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:03.719795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed89ea7-1b3c-4b70-8692-bf11505e7fc2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:03.720628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed89ea7-1b3c-4b70-8692-bf11505e7fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:03.720628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed89ea7-1b3c-4b70-8692-bf11505e7fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:03.743526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6ed89ea7-1b3c-4b70-8692-bf11505e7fc2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:03.745000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6ed89ea7-1b3c-4b70-8692-bf11505e7fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:03.837307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1069/2155] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:03.861285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296af6eb-a8fb-4cea-ad6b-2925ab25878a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:03.861285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296af6eb-a8fb-4cea-ad6b-2925ab25878a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:03.861285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296af6eb-a8fb-4cea-ad6b-2925ab25878a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:03.866613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-296af6eb-a8fb-4cea-ad6b-2925ab25878a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:03.867840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-296af6eb-a8fb-4cea-ad6b-2925ab25878a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:03.877744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1087/2156] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:03.882423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed89ea7-1b3c-4b70-8692-bf11505e7fc2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:03.894424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed89ea7-1b3c-4b70-8692-bf11505e7fc2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:03.901043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:03.901676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:03.901963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:03.902250 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:03.902921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:40:03.903225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:03.905112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:03.927183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1070/2157] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:40:03.933085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296af6eb-a8fb-4cea-ad6b-2925ab25878a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:03.946282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296af6eb-a8fb-4cea-ad6b-2925ab25878a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:03.951850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d359f37-f23b-4d80-953d-8057e8611be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:03.952519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d359f37-f23b-4d80-953d-8057e8611be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:03.952796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d359f37-f23b-4d80-953d-8057e8611be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:03.953069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d359f37-f23b-4d80-953d-8057e8611be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:03.953735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d359f37-f23b-4d80-953d-8057e8611be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:03.955694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d359f37-f23b-4d80-953d-8057e8611be2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:04.126682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1088/2158] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:03 2022] POST /identity/v3/projects => generated 393 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:04.132023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:04.143509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036e7d8-72d1-4389-9685-7fd838a1e3bd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:04.264382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c911e94-33d5-487b-bcc2-418b674e0055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:04.265541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c911e94-33d5-487b-bcc2-418b674e0055 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:04.265989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c911e94-33d5-487b-bcc2-418b674e0055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:04.266578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c911e94-33d5-487b-bcc2-418b674e0055 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:04.267386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c911e94-33d5-487b-bcc2-418b674e0055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:04.271680 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c911e94-33d5-487b-bcc2-418b674e0055 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:04.380786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:04.388037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1071/2159] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:04 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:04.393686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d359f37-f23b-4d80-953d-8057e8611be2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:04.406974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d359f37-f23b-4d80-953d-8057e8611be2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:04.412887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62151f9-952c-4cbf-a02c-8620b3f0626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:04.413529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62151f9-952c-4cbf-a02c-8620b3f0626b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:04.413817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62151f9-952c-4cbf-a02c-8620b3f0626b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:04.414155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62151f9-952c-4cbf-a02c-8620b3f0626b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:04.414892 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62151f9-952c-4cbf-a02c-8620b3f0626b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:04.416763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62151f9-952c-4cbf-a02c-8620b3f0626b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:04.431285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1089/2160] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:40:04.438279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c911e94-33d5-487b-bcc2-418b674e0055 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:04.454491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c911e94-33d5-487b-bcc2-418b674e0055 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:04.459924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:04.460661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:04.460945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:04.461224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:04.461897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:04.463659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:04.482749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1072/2161] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:04.484885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62151f9-952c-4cbf-a02c-8620b3f0626b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:04.520752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:04.524522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:04.526607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62151f9-952c-4cbf-a02c-8620b3f0626b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:04.536479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb9268-aac6-443a-b5c7-c56181c1059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:04.538182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb9268-aac6-443a-b5c7-c56181c1059a admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:04.540309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb9268-aac6-443a-b5c7-c56181c1059a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:04.541077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb9268-aac6-443a-b5c7-c56181c1059a admin admin] PATH_INFO: `/v3/projects/127e5da55b164ce68568c153fee58baf/users/adb792e7246d42dface4aa52899a0ffc/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:04.556704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2150-564d-4a49-b244-8cfef180fb07 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:04.583328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:04.584475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:04.584995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:04.585474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] PATH_INFO: `/v3/users/20c258b3ad7340acbe00bc4c790880a0` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:04.588444 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20c258b3ad7340acbe00bc4c790880a0)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:04.601327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:04.608928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:04.611477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb9268-aac6-443a-b5c7-c56181c1059a admin admin] RBAC: Authorizing `identity:create_grant(project_id=127e5da55b164ce68568c153fee58baf, user_id=adb792e7246d42dface4aa52899a0ffc, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:04.632500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb9268-aac6-443a-b5c7-c56181c1059a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:04.735221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1090/2162] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:04 2022] PUT /identity/v3/projects/127e5da55b164ce68568c153fee58baf/users/adb792e7246d42dface4aa52899a0ffc/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:04.737970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e886b51-27dc-4b3c-b183-b78f159f21c8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:04.738847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e886b51-27dc-4b3c-b183-b78f159f21c8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:04.739609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e886b51-27dc-4b3c-b183-b78f159f21c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:04.794498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0e886b51-27dc-4b3c-b183-b78f159f21c8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:04.800274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0e886b51-27dc-4b3c-b183-b78f159f21c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:04.897789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1091/2163] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:04.924949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e886b51-27dc-4b3c-b183-b78f159f21c8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:04.926597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0e886b51-27dc-4b3c-b183-b78f159f21c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:04.932821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20c258b3ad7340acbe00bc4c790880a0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:04.935525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20c258b3ad7340acbe00bc4c790880a0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:04.957509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20c258b3ad7340acbe00bc4c790880a0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:04.959482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20c258b3ad7340acbe00bc4c790880a0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:05.016049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e886b51-27dc-4b3c-b183-b78f159f21c8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:05.020349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725423a-d590-45f9-80f4-d081d5a97acc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:05.020972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725423a-d590-45f9-80f4-d081d5a97acc service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:05.021257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725423a-d590-45f9-80f4-d081d5a97acc service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:05.021530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725423a-d590-45f9-80f4-d081d5a97acc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:05.023473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b725423a-d590-45f9-80f4-d081d5a97acc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:05.037866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725423a-d590-45f9-80f4-d081d5a97acc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:05.039352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725423a-d590-45f9-80f4-d081d5a97acc service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:05.051321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1092/2164] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:05.057968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20c258b3ad7340acbe00bc4c790880a0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:05.090986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1073/2165] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:04 2022] DELETE /identity/v3/users/20c258b3ad7340acbe00bc4c790880a0 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:40:05.094803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec2ffa-995c-4ce2-a248-4f7c8450665a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:05.100482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec2ffa-995c-4ce2-a248-4f7c8450665a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:05.100482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec2ffa-995c-4ce2-a248-4f7c8450665a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:05.126013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-86ec2ffa-995c-4ce2-a248-4f7c8450665a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:05.127244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-86ec2ffa-995c-4ce2-a248-4f7c8450665a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:05.240401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1093/2166] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:05.252546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:05.255883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:05.368404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4329c3f-ca9f-4a27-8450-b1af10cba488 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:05.392856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:05.392856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:05.392856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:05.392856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:05.405337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:05.407052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:05.419979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1074/2167] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:05.846592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec2ffa-995c-4ce2-a248-4f7c8450665a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:05.848914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-86ec2ffa-995c-4ce2-a248-4f7c8450665a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:05.863109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec2ffa-995c-4ce2-a248-4f7c8450665a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:05.868296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:05.868938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:05.869223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:05.869500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] PATH_INFO: `/v3/users/a4a4f64021854a36ad52ee1e25c0306a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:05.870774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a4f64021854a36ad52ee1e25c0306a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:05.872962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:05.910393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:05.913904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:05.936713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70ab9e-d181-4045-be4f-3e30a2cd68ef service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:05.947721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:05.948523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:05.949237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:05.949719 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] PATH_INFO: `/v3/projects/70d0978c718d4704905e96d4ef197796` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:05.951611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=70d0978c718d4704905e96d4ef197796)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:05.954809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:06.113286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a4f64021854a36ad52ee1e25c0306a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.114084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a4f64021854a36ad52ee1e25c0306a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.124238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70d0978c718d4704905e96d4ef197796', 'request_id': 'req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.130502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a4f64021854a36ad52ee1e25c0306a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.133134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70d0978c718d4704905e96d4ef197796', 'request_id': 'req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.141708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70d0978c718d4704905e96d4ef197796', 'request_id': 'req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.266660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a4f64021854a36ad52ee1e25c0306a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.267388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9e884423-d8cf-42ce-89a5-2f324eb7a55a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a4f64021854a36ad52ee1e25c0306a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.296951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1094/2168] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:05 2022] DELETE /identity/v3/users/a4a4f64021854a36ad52ee1e25c0306a => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:06.301874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126db07-3cbd-4aa3-9032-1c08998002a6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.302166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126db07-3cbd-4aa3-9032-1c08998002a6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.302441 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126db07-3cbd-4aa3-9032-1c08998002a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.322627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-6126db07-3cbd-4aa3-9032-1c08998002a6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:06.323831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6126db07-3cbd-4aa3-9032-1c08998002a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:06.373397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] The token cache is being invalidate because project 70d0978c718d4704905e96d4ef197796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:06.373816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.374601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7a9a1459-b25d-4843-87d1-43de3d9cfc1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:06.376286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1075/2169] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:05 2022] DELETE /identity/v3/projects/70d0978c718d4704905e96d4ef197796 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:06.381070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5cd0-35c6-4f85-b022-73131ce19cb7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.381364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5cd0-35c6-4f85-b022-73131ce19cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.381651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5cd0-35c6-4f85-b022-73131ce19cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.383417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1095/2170] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:40:06.394046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126db07-3cbd-4aa3-9032-1c08998002a6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:06.395454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6126db07-3cbd-4aa3-9032-1c08998002a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:06.408906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ee9e5cd0-35c6-4f85-b022-73131ce19cb7 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:06.410168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ee9e5cd0-35c6-4f85-b022-73131ce19cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:06.418803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1076/2171] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:40:06.427948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5acc4-c728-4259-8205-007587844f9d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.428283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5acc4-c728-4259-8205-007587844f9d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.428566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5acc4-c728-4259-8205-007587844f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.450427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4ab5acc4-c728-4259-8205-007587844f9d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:06.451472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4ab5acc4-c728-4259-8205-007587844f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:06.463266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126db07-3cbd-4aa3-9032-1c08998002a6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:06.465597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1077/2172] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:06.469679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31b614-9340-4996-858a-bb9eeb98fd53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:06.470309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31b614-9340-4996-858a-bb9eeb98fd53 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.470713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31b614-9340-4996-858a-bb9eeb98fd53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.470904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31b614-9340-4996-858a-bb9eeb98fd53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.472832 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5acc4-c728-4259-8205-007587844f9d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:06.483822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5acc4-c728-4259-8205-007587844f9d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:06.486459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:06.489736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31b614-9340-4996-858a-bb9eeb98fd53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:06.491407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31b614-9340-4996-858a-bb9eeb98fd53 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:06.492489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:06.493410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.493872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.494470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.495365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:40:06.495812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:06.497636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:06.503304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1096/2173] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:06.516229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1078/2174] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:40:06.517891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31b614-9340-4996-858a-bb9eeb98fd53 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:06.529165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31b614-9340-4996-858a-bb9eeb98fd53 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:06.535067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:06.535987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.536742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.536742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.537244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:06.539189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:06.656920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1097/2175] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:06 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:06.657768 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:06.668668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c37884-28c9-4586-b7d2-285ede746be4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:06.710567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:06.730335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899f07-cd98-4af8-8e0c-dd6cab8e61be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:06.731124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899f07-cd98-4af8-8e0c-dd6cab8e61be admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.731410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899f07-cd98-4af8-8e0c-dd6cab8e61be admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.731693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c899f07-cd98-4af8-8e0c-dd6cab8e61be admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.732364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c899f07-cd98-4af8-8e0c-dd6cab8e61be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:06.735161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c899f07-cd98-4af8-8e0c-dd6cab8e61be admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:06.785646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:06.787537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:06.801548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68638c-28c6-4632-a9c0-e532a141a61d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:06.806372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:06.807095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.807396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.807639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] PATH_INFO: `/v3/users/2475da9c30184832bb205aac12ec6840` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.809082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2475da9c30184832bb205aac12ec6840)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:06.811054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:06.922359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1079/2176] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:06 2022] POST /identity/v3/users => generated 287 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:06.927348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899f07-cd98-4af8-8e0c-dd6cab8e61be admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:06.942213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c899f07-cd98-4af8-8e0c-dd6cab8e61be admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:06.947805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268cb20-e0ef-496f-ba08-9e8a962d3829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:06.948981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268cb20-e0ef-496f-ba08-9e8a962d3829 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.949445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268cb20-e0ef-496f-ba08-9e8a962d3829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.949964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268cb20-e0ef-496f-ba08-9e8a962d3829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.950584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268cb20-e0ef-496f-ba08-9e8a962d3829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:06.952858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268cb20-e0ef-496f-ba08-9e8a962d3829 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:06.968927 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1080/2177] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:06.973378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268cb20-e0ef-496f-ba08-9e8a962d3829 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:06.987161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268cb20-e0ef-496f-ba08-9e8a962d3829 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:06.996132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdf87f-0f1d-493f-bfe9-30bb157b8ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:06.997153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdf87f-0f1d-493f-bfe9-30bb157b8ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:06.997543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdf87f-0f1d-493f-bfe9-30bb157b8ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:06.997818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdf87f-0f1d-493f-bfe9-30bb157b8ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:06.998566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdf87f-0f1d-493f-bfe9-30bb157b8ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:07.000597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdf87f-0f1d-493f-bfe9-30bb157b8ee6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:07.016750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1081/2178] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:40:07.021732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdf87f-0f1d-493f-bfe9-30bb157b8ee6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:07.032522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2475da9c30184832bb205aac12ec6840'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.034709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2475da9c30184832bb205aac12ec6840'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.040501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdf87f-0f1d-493f-bfe9-30bb157b8ee6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:07.045915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d04f64-6898-49e0-af40-b83a48a15f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:07.047079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d04f64-6898-49e0-af40-b83a48a15f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:07.047418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d04f64-6898-49e0-af40-b83a48a15f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:07.047695 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d04f64-6898-49e0-af40-b83a48a15f9d admin admin] PATH_INFO: `/v3/projects/fe1820eda32d4501a64dfaa630241614/users/055197d479674f8290c0808baf32180f/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:07.052979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2475da9c30184832bb205aac12ec6840'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.070124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d04f64-6898-49e0-af40-b83a48a15f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1820eda32d4501a64dfaa630241614, user_id=055197d479674f8290c0808baf32180f, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:07.073500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d04f64-6898-49e0-af40-b83a48a15f9d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:07.208939 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2475da9c30184832bb205aac12ec6840'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.211050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d934b76c-5e80-472f-854a-39176e28f250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2475da9c30184832bb205aac12ec6840'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.239603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1098/2180] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:06 2022] DELETE /identity/v3/users/2475da9c30184832bb205aac12ec6840 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:40:07.240140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1082/2180] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:07 2022] PUT /identity/v3/projects/fe1820eda32d4501a64dfaa630241614/users/055197d479674f8290c0808baf32180f/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:07.244714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00a50b-221c-46cf-9858-9684235e3d1c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:07.245374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00a50b-221c-46cf-9858-9684235e3d1c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:07.245857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00a50b-221c-46cf-9858-9684235e3d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:07.256190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525be56-9fa9-47e2-afdb-da7949c5cf88 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:07.256514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525be56-9fa9-47e2-afdb-da7949c5cf88 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:07.257762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525be56-9fa9-47e2-afdb-da7949c5cf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:07.289399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2a00a50b-221c-46cf-9858-9684235e3d1c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:07.290685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2a00a50b-221c-46cf-9858-9684235e3d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:07.323922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4525be56-9fa9-47e2-afdb-da7949c5cf88 None None] MFA Rules not processed for user `055197d479674f8290c0808baf32180f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:07.325583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4525be56-9fa9-47e2-afdb-da7949c5cf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:07.407386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1083/2181] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:40:07.408701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00a50b-221c-46cf-9858-9684235e3d1c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:07.410654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2a00a50b-221c-46cf-9858-9684235e3d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:07.427167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00a50b-221c-46cf-9858-9684235e3d1c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:07.440481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:07.441677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:07.442248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:07.442601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] PATH_INFO: `/v3/users/b943816026cb4aa49c82f555fc115e2b` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:07.444373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b943816026cb4aa49c82f555fc115e2b)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:07.447994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:07.513061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1099/2182] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:07.513721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525be56-9fa9-47e2-afdb-da7949c5cf88 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:07.529560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4525be56-9fa9-47e2-afdb-da7949c5cf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:07.647829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525be56-9fa9-47e2-afdb-da7949c5cf88 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:07.657301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:07.658119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:07.658534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:07.658955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:07.680352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:07.696680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:07.715320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1100/2183] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:07.717125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:07.720167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:07.744116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29272d1b-2e54-480d-aedd-eca10464ad07 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:07.753836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:07.776371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:07.778727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:07.779410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:07.780604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] PATH_INFO: `/v3/projects/a26934f04fae43ffbad78bcf9108f8cf` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:07.783248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26934f04fae43ffbad78bcf9108f8cf)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:07.789728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:07.803672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b943816026cb4aa49c82f555fc115e2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.804163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b943816026cb4aa49c82f555fc115e2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.833735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b943816026cb4aa49c82f555fc115e2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.833735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b943816026cb4aa49c82f555fc115e2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.918818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b943816026cb4aa49c82f555fc115e2b'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:07.962672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1084/2184] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:07 2022] DELETE /identity/v3/users/b943816026cb4aa49c82f555fc115e2b => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:07.965511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:07.969152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:07.993860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26934f04fae43ffbad78bcf9108f8cf', 'request_id': 'req-127a471c-aaf1-400e-9a48-80d09c021e4f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.005316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26934f04fae43ffbad78bcf9108f8cf', 'request_id': 'req-127a471c-aaf1-400e-9a48-80d09c021e4f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.077583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd28d91-050b-43de-9b17-8aa7fff5bfef admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:08.081978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9662fe9-fa4a-4fce-959a-7102c70e71ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:08.082749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9662fe9-fa4a-4fce-959a-7102c70e71ba service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:08.082868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9662fe9-fa4a-4fce-959a-7102c70e71ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:08.083147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9662fe9-fa4a-4fce-959a-7102c70e71ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:08.085030 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b9662fe9-fa4a-4fce-959a-7102c70e71ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:08.159085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26934f04fae43ffbad78bcf9108f8cf', 'request_id': 'req-127a471c-aaf1-400e-9a48-80d09c021e4f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.160929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9662fe9-fa4a-4fce-959a-7102c70e71ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:08.162477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9662fe9-fa4a-4fce-959a-7102c70e71ba service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:08.173258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1085/2185] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:08.174741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e461b7-95c5-4b2b-adda-24cf1a6d1d5b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:08.175342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e461b7-95c5-4b2b-adda-24cf1a6d1d5b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:08.175342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e461b7-95c5-4b2b-adda-24cf1a6d1d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:08.198993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] The token cache is being invalidate because project a26934f04fae43ffbad78bcf9108f8cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:08.199251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-49e461b7-95c5-4b2b-adda-24cf1a6d1d5b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:08.200291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.200740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-49e461b7-95c5-4b2b-adda-24cf1a6d1d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:08.204458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.208338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1101/2186] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:07 2022] DELETE /identity/v3/projects/a26934f04fae43ffbad78bcf9108f8cf => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:08.212270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:08.215420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:08.349301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1086/2187] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:08.349301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e461b7-95c5-4b2b-adda-24cf1a6d1d5b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:08.349542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a471c-aaf1-400e-9a48-80d09c021e4f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:08.351348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-49e461b7-95c5-4b2b-adda-24cf1a6d1d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:08.353118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:08.353937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:08.354230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:08.354517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] PATH_INFO: `/v3/users/3cf8ac51fa3c45e195c94f498148b18a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:08.355974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf8ac51fa3c45e195c94f498148b18a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:08.358260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:08.368575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e461b7-95c5-4b2b-adda-24cf1a6d1d5b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:08.378581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:08.379308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:08.379578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:08.380066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] PATH_INFO: `/v3/users/055197d479674f8290c0808baf32180f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:08.382520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=055197d479674f8290c0808baf32180f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:08.384845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:08.700246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055197d479674f8290c0808baf32180f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.701424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf8ac51fa3c45e195c94f498148b18a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.702288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055197d479674f8290c0808baf32180f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.702864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf8ac51fa3c45e195c94f498148b18a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.713306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf8ac51fa3c45e195c94f498148b18a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.715551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055197d479674f8290c0808baf32180f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.715551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055197d479674f8290c0808baf32180f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.797827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf8ac51fa3c45e195c94f498148b18a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.798598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf8ac51fa3c45e195c94f498148b18a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.821898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1102/2188] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:08 2022] DELETE /identity/v3/users/3cf8ac51fa3c45e195c94f498148b18a => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:08.822645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:08.824836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:08.890466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055197d479674f8290c0808baf32180f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:08.916165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1087/2189] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:08 2022] DELETE /identity/v3/users/055197d479674f8290c0808baf32180f => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:08.916165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:08.922993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:08.939047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25da8e7-29f7-43c1-83f8-5a8149334edc admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:09.007172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5857b1-ce8d-4c44-bb4c-78444991fe9d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:09.012206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:09.012929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.013282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.013650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] PATH_INFO: `/v3/projects/79f8a4e5bb274136b40fdfce9300be78` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.014956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f8a4e5bb274136b40fdfce9300be78)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:09.015294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131aad2-c727-40bf-87f9-2b4d2dee3c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:09.015912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131aad2-c727-40bf-87f9-2b4d2dee3c39 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.016215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131aad2-c727-40bf-87f9-2b4d2dee3c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.016499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131aad2-c727-40bf-87f9-2b4d2dee3c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.017392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:09.018324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c131aad2-c727-40bf-87f9-2b4d2dee3c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:09.033086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131aad2-c727-40bf-87f9-2b4d2dee3c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:09.034813 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131aad2-c727-40bf-87f9-2b4d2dee3c39 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:09.046493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1103/2190] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:09.047719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5a9ad-0a22-43ba-a8eb-2025d8374208 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.048005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5a9ad-0a22-43ba-a8eb-2025d8374208 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.048304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5a9ad-0a22-43ba-a8eb-2025d8374208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.071099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8cc5a9ad-0a22-43ba-a8eb-2025d8374208 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:09.072331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8cc5a9ad-0a22-43ba-a8eb-2025d8374208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:09.081280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1104/2191] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:09.095139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5a9ad-0a22-43ba-a8eb-2025d8374208 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:09.107167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5a9ad-0a22-43ba-a8eb-2025d8374208 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:09.113748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f616dd-c746-4c22-acd6-c9ae31a28872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:09.114426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f616dd-c746-4c22-acd6-c9ae31a28872 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.114716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f616dd-c746-4c22-acd6-c9ae31a28872 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.114990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f616dd-c746-4c22-acd6-c9ae31a28872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.128310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f616dd-c746-4c22-acd6-c9ae31a28872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:09.130493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f616dd-c746-4c22-acd6-c9ae31a28872 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:09.150438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f8a4e5bb274136b40fdfce9300be78', 'request_id': 'req-2a29f85f-4f45-4868-98c8-1947627d4412'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:09.151072 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1105/2192] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:09.160233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f8a4e5bb274136b40fdfce9300be78', 'request_id': 'req-2a29f85f-4f45-4868-98c8-1947627d4412'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:09.169169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f8a4e5bb274136b40fdfce9300be78', 'request_id': 'req-2a29f85f-4f45-4868-98c8-1947627d4412'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:09.301346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] The token cache is being invalidate because project 79f8a4e5bb274136b40fdfce9300be78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:09.301929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:09.303429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2a29f85f-4f45-4868-98c8-1947627d4412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:09.308912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1088/2193] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:08 2022] DELETE /identity/v3/projects/79f8a4e5bb274136b40fdfce9300be78 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:09.317136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71203d98-3954-4bc3-a816-970512b4e9e8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.317696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71203d98-3954-4bc3-a816-970512b4e9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.317855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71203d98-3954-4bc3-a816-970512b4e9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.349986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-71203d98-3954-4bc3-a816-970512b4e9e8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:09.351310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-71203d98-3954-4bc3-a816-970512b4e9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:09.502212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1106/2194] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:09.511475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e23a4-577c-4cdf-a904-769176da5171 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.511774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e23a4-577c-4cdf-a904-769176da5171 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.512075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e23a4-577c-4cdf-a904-769176da5171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.528495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:09.551711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-994e23a4-577c-4cdf-a904-769176da5171 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:09.553024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-994e23a4-577c-4cdf-a904-769176da5171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:09.577765 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1089/2195] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:09.582194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71203d98-3954-4bc3-a816-970512b4e9e8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:09.594083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71203d98-3954-4bc3-a816-970512b4e9e8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:09.601251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:09.602408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.602772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.603393 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.604940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:40:09.606039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:09.610424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:09.631624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1107/2196] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:40:09.632582 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e23a4-577c-4cdf-a904-769176da5171 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:09.633517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:09.634999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-994e23a4-577c-4cdf-a904-769176da5171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:09.649135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe85e8-4a3e-4014-8f93-df55c1e0c197 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:09.650391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e23a4-577c-4cdf-a904-769176da5171 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:09.654369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f133d6-4e86-4331-bac9-872bf8d40785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:09.656654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f133d6-4e86-4331-bac9-872bf8d40785 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.657121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f133d6-4e86-4331-bac9-872bf8d40785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.657467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f133d6-4e86-4331-bac9-872bf8d40785 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.658121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f133d6-4e86-4331-bac9-872bf8d40785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:09.660150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f133d6-4e86-4331-bac9-872bf8d40785 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:09.663347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:09.664260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:09.665146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:09.665615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] PATH_INFO: `/v3/projects/d856b049b2244fb7a87e5179f3d45cd6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:09.668337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d856b049b2244fb7a87e5179f3d45cd6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:09.677714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:09.817534 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1108/2197] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:09 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:09.822407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f133d6-4e86-4331-bac9-872bf8d40785 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:09.966230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd856b049b2244fb7a87e5179f3d45cd6', 'request_id': 'req-3a77ea21-6290-411c-b864-2c0bbf28b24a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:09.976123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd856b049b2244fb7a87e5179f3d45cd6', 'request_id': 'req-3a77ea21-6290-411c-b864-2c0bbf28b24a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:09.976501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f133d6-4e86-4331-bac9-872bf8d40785 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:09.985695 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd856b049b2244fb7a87e5179f3d45cd6', 'request_id': 'req-3a77ea21-6290-411c-b864-2c0bbf28b24a'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.118803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] The token cache is being invalidate because project d856b049b2244fb7a87e5179f3d45cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:10.126700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.128345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.134880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1090/2198] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:09 2022] DELETE /identity/v3/projects/d856b049b2244fb7a87e5179f3d45cd6 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:10.136761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:10.143149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:10.162004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:10.163709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:10.163709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:10.163709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:10.164002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:10.165993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:10.228958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77ea21-6290-411c-b864-2c0bbf28b24a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:10.241175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:10.241828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:10.242112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:10.243175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] PATH_INFO: `/v3/projects/fa992541102c467c92cff50ae45186b8` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:10.244763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa992541102c467c92cff50ae45186b8)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:10.247575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:10.335194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1109/2199] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:09 2022] POST /identity/v3/users => generated 292 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:10.336281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:10.343228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:10.357485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f751e41-f7cd-4cf0-ad20-9bd2dfa2f1ae admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:10.362646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:10.363380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:10.363760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:10.364109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] PATH_INFO: `/v3/projects/fe1820eda32d4501a64dfaa630241614` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:10.365500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1820eda32d4501a64dfaa630241614)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:10.367455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:10.441858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa992541102c467c92cff50ae45186b8', 'request_id': 'req-669db2ef-f408-4465-8401-03005509cd23'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.451258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa992541102c467c92cff50ae45186b8', 'request_id': 'req-669db2ef-f408-4465-8401-03005509cd23'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.461414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa992541102c467c92cff50ae45186b8', 'request_id': 'req-669db2ef-f408-4465-8401-03005509cd23'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.557063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] The token cache is being invalidate because project fa992541102c467c92cff50ae45186b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:10.557473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.560380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.575061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1091/2200] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:10 2022] DELETE /identity/v3/projects/fa992541102c467c92cff50ae45186b8 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:10.575244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1820eda32d4501a64dfaa630241614', 'request_id': 'req-f5100f79-9979-40f6-9a16-714489e3d7c6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.575343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:10.587840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1820eda32d4501a64dfaa630241614', 'request_id': 'req-f5100f79-9979-40f6-9a16-714489e3d7c6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.587982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:10.646889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:10.720149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1820eda32d4501a64dfaa630241614', 'request_id': 'req-f5100f79-9979-40f6-9a16-714489e3d7c6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.759066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] The token cache is being invalidate because project fe1820eda32d4501a64dfaa630241614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:10.759562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.760682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:10.762947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1110/2201] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:10 2022] DELETE /identity/v3/projects/fe1820eda32d4501a64dfaa630241614 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:10.783196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:10.794785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:10.798478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db2ef-f408-4465-8401-03005509cd23 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:10.921614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5100f79-9979-40f6-9a16-714489e3d7c6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:10.928634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:10.929271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:10.929558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:10.929838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:10.930513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:10.932332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:10.950695 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1111/2202] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:10.952280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:10.954571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:10.970447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977ab9c-c7ba-432d-bb40-85eb6e90b464 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:10.977163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:10.978151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:10.978835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:10.979306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] PATH_INFO: `/v3/users/d961a71bbc064be09d2b289bbf40aa67` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:10.981720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d961a71bbc064be09d2b289bbf40aa67)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:10.984651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:11.057544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:11.058224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:11.058764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:11.059052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] PATH_INFO: `/v3/users/605b5229718d44c5bd25e52d1af28264` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:11.060403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=605b5229718d44c5bd25e52d1af28264)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:11.063691 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:11.344095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd961a71bbc064be09d2b289bbf40aa67'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.346554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd961a71bbc064be09d2b289bbf40aa67'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.357149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd961a71bbc064be09d2b289bbf40aa67'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.363639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '605b5229718d44c5bd25e52d1af28264'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.364886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '605b5229718d44c5bd25e52d1af28264'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.379614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '605b5229718d44c5bd25e52d1af28264'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.380848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '605b5229718d44c5bd25e52d1af28264'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.546438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd961a71bbc064be09d2b289bbf40aa67'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.546438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd961a71bbc064be09d2b289bbf40aa67'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.565021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1112/2203] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:10 2022] DELETE /identity/v3/users/d961a71bbc064be09d2b289bbf40aa67 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:11.566275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:11.569009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:11.649694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '605b5229718d44c5bd25e52d1af28264'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:11.677721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1092/2204] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:10 2022] DELETE /identity/v3/users/605b5229718d44c5bd25e52d1af28264 => generated 0 bytes in 1103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:11.685710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98d4f-8ff7-4007-8115-b11af443e3fa admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:11.776712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb65e3-f564-4110-94ef-db63677db09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:11.777343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb65e3-f564-4110-94ef-db63677db09c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:11.777622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb65e3-f564-4110-94ef-db63677db09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:11.777898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb65e3-f564-4110-94ef-db63677db09c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:11.778705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb65e3-f564-4110-94ef-db63677db09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:11.781209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb65e3-f564-4110-94ef-db63677db09c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:11.793388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1113/2205] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:11.798354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:11.800038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:11.814966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b458d83-a1c2-4f3b-ab09-ad4db0514c1e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:11.821784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce50157-b569-4a15-8ef4-a65fdd0dd866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:11.822660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce50157-b569-4a15-8ef4-a65fdd0dd866 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:11.823107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce50157-b569-4a15-8ef4-a65fdd0dd866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:11.823578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce50157-b569-4a15-8ef4-a65fdd0dd866 admin admin] PATH_INFO: `/v3/projects/609ef41523c24dc09131a8a8ac829805/users/d56decda97274523955b0c38d1cae646/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:11.844973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce50157-b569-4a15-8ef4-a65fdd0dd866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609ef41523c24dc09131a8a8ac829805, user_id=d56decda97274523955b0c38d1cae646, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:11.847763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce50157-b569-4a15-8ef4-a65fdd0dd866 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:11.995845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1093/2206] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:11 2022] PUT /identity/v3/projects/609ef41523c24dc09131a8a8ac829805/users/d56decda97274523955b0c38d1cae646/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:40:12.007691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82af79-60d0-4ca2-bab5-925f52297311 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:12.007989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82af79-60d0-4ca2-bab5-925f52297311 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:12.008307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82af79-60d0-4ca2-bab5-925f52297311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:12.066418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8e82af79-60d0-4ca2-bab5-925f52297311 None None] MFA Rules not processed for user `d56decda97274523955b0c38d1cae646`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:12.067672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8e82af79-60d0-4ca2-bab5-925f52297311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:12.237075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1114/2207] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:12.241725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce50157-b569-4a15-8ef4-a65fdd0dd866 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:12.333581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce50157-b569-4a15-8ef4-a65fdd0dd866 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:12.339417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c113b-1884-49cd-862a-b93c0758bfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:12.340280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c113b-1884-49cd-862a-b93c0758bfe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:12.340636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c113b-1884-49cd-862a-b93c0758bfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:12.341330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c113b-1884-49cd-862a-b93c0758bfe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:12.342857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c113b-1884-49cd-862a-b93c0758bfe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:12.345330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c113b-1884-49cd-862a-b93c0758bfe6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:12.408910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82af79-60d0-4ca2-bab5-925f52297311 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:12.411191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8e82af79-60d0-4ca2-bab5-925f52297311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:12.432233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1094/2208] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:12 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:40:12.435536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c113b-1884-49cd-862a-b93c0758bfe6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:12.447895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82af79-60d0-4ca2-bab5-925f52297311 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:12.640816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c113b-1884-49cd-862a-b93c0758bfe6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:12.646634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:12.647286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:12.647570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:12.648919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] PATH_INFO: `/v3/projects/e175c4cc812548db8b60c07d1b9c8f9d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:12.650283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e175c4cc812548db8b60c07d1b9c8f9d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:12.657771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:12.664254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:12.664901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:12.665189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:12.665471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:12.666113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:12.668150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:12.786610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1095/2209] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:12 2022] POST /identity/v3/users => generated 293 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:12.787744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:12.789725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:12.804371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2d997-92e1-498a-aacc-31518e7b8bb8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:12.814828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:12.815615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:12.815882 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:12.816270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] PATH_INFO: `/v3/projects/87e79ae44b7041f8ae136316943f0b1e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:12.817805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87e79ae44b7041f8ae136316943f0b1e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:12.819971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:12.876688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e175c4cc812548db8b60c07d1b9c8f9d', 'request_id': 'req-724492a1-8494-44f7-83ca-7e3a752f3735'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:12.884484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e175c4cc812548db8b60c07d1b9c8f9d', 'request_id': 'req-724492a1-8494-44f7-83ca-7e3a752f3735'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:12.996428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e175c4cc812548db8b60c07d1b9c8f9d', 'request_id': 'req-724492a1-8494-44f7-83ca-7e3a752f3735'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:12.997450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:13.011586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] The token cache is being invalidate because project e175c4cc812548db8b60c07d1b9c8f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:13.012345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.013028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.013175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87e79ae44b7041f8ae136316943f0b1e', 'request_id': 'req-d27b191f-b021-4f45-871b-fd5c5ffd6e16'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.020010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1115/2210] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:12 2022] DELETE /identity/v3/projects/e175c4cc812548db8b60c07d1b9c8f9d => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:40:13.021678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87e79ae44b7041f8ae136316943f0b1e', 'request_id': 'req-d27b191f-b021-4f45-871b-fd5c5ffd6e16'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.028740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:13.031779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:13.040054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87e79ae44b7041f8ae136316943f0b1e', 'request_id': 'req-d27b191f-b021-4f45-871b-fd5c5ffd6e16'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.131953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724492a1-8494-44f7-83ca-7e3a752f3735 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:13.167341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] The token cache is being invalidate because project 87e79ae44b7041f8ae136316943f0b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:13.167939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.169035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.172261 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1096/2211] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:12 2022] DELETE /identity/v3/projects/87e79ae44b7041f8ae136316943f0b1e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:13.173443 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:13.175017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:13.224005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62c7e8-dac0-4a38-bfab-60abef108202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:13.224738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62c7e8-dac0-4a38-bfab-60abef108202 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:13.225121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62c7e8-dac0-4a38-bfab-60abef108202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:13.225409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62c7e8-dac0-4a38-bfab-60abef108202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:13.226075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62c7e8-dac0-4a38-bfab-60abef108202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:13.227717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62c7e8-dac0-4a38-bfab-60abef108202 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:13.249793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1116/2212] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:13.249793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c56206-1be2-4c6c-9915-4e9257da3682 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:13.249793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c56206-1be2-4c6c-9915-4e9257da3682 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:13.249793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c56206-1be2-4c6c-9915-4e9257da3682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:13.254390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b191f-b021-4f45-871b-fd5c5ffd6e16 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:13.266544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:13.267408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:13.267724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:13.268057 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] PATH_INFO: `/v3/users/d8b6313ca6f6402b80783b2622ea41c6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:13.269928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b6313ca6f6402b80783b2622ea41c6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:13.272167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:13.293041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-41c56206-1be2-4c6c-9915-4e9257da3682 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:13.296723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41c56206-1be2-4c6c-9915-4e9257da3682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:13.310166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1117/2213] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:13.311645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c56206-1be2-4c6c-9915-4e9257da3682 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:13.313280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-41c56206-1be2-4c6c-9915-4e9257da3682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:13.326990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c56206-1be2-4c6c-9915-4e9257da3682 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:13.333237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf169758-0008-4d8d-8046-8b78b5b135cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:13.334340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf169758-0008-4d8d-8046-8b78b5b135cb admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:13.334880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf169758-0008-4d8d-8046-8b78b5b135cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:13.335376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf169758-0008-4d8d-8046-8b78b5b135cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:13.336886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf169758-0008-4d8d-8046-8b78b5b135cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:13.339833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf169758-0008-4d8d-8046-8b78b5b135cb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:13.367356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1118/2214] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:13.367356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e319219-c52e-4098-b834-0e277c0590cc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:13.367619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e319219-c52e-4098-b834-0e277c0590cc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:13.368093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e319219-c52e-4098-b834-0e277c0590cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:13.408144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1e319219-c52e-4098-b834-0e277c0590cc None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:13.411156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1e319219-c52e-4098-b834-0e277c0590cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:13.446356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1119/2215] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:13.449827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e319219-c52e-4098-b834-0e277c0590cc None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:13.459428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e319219-c52e-4098-b834-0e277c0590cc None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:13.471959 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:13.472764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:13.479304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:13.479759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] PATH_INFO: `/v3/projects/09be5b26299442109a350b9d82b94887/users/7fc5b0225c914dc9bc3c64ffc53f1324/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:13.507538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09be5b26299442109a350b9d82b94887, user_id=7fc5b0225c914dc9bc3c64ffc53f1324, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:13.511075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:13.575310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b6313ca6f6402b80783b2622ea41c6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.577293 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b6313ca6f6402b80783b2622ea41c6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.601271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1120/2216] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:13 2022] PUT /identity/v3/projects/09be5b26299442109a350b9d82b94887/users/7fc5b0225c914dc9bc3c64ffc53f1324/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:13.602212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b6313ca6f6402b80783b2622ea41c6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.603573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:13.604087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b6313ca6f6402b80783b2622ea41c6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.612465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:13.721971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3ca573ce-2490-481d-9e3d-846dd1b5bf78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b6313ca6f6402b80783b2622ea41c6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:13.790977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1097/2217] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:13 2022] DELETE /identity/v3/users/d8b6313ca6f6402b80783b2622ea41c6 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:13.792747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d263734-4854-4b19-b635-3ea2dcb344f4 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:13.793081 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d263734-4854-4b19-b635-3ea2dcb344f4 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:13.793398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d263734-4854-4b19-b635-3ea2dcb344f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:13.867836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb314f9-2c08-4205-91b0-c5dc252ab362 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:13.895511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8d263734-4854-4b19-b635-3ea2dcb344f4 None None] MFA Rules not processed for user `7fc5b0225c914dc9bc3c64ffc53f1324`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:13.896803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8d263734-4854-4b19-b635-3ea2dcb344f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:14.000712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c37bee-a634-4c48-ae09-381e8cfdad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.001392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c37bee-a634-4c48-ae09-381e8cfdad47 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.001673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c37bee-a634-4c48-ae09-381e8cfdad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.001945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c37bee-a634-4c48-ae09-381e8cfdad47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.002650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c37bee-a634-4c48-ae09-381e8cfdad47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:40:14.002955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c37bee-a634-4c48-ae09-381e8cfdad47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.004663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c37bee-a634-4c48-ae09-381e8cfdad47 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.019683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1121/2218] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:40:14.021183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14732a-d796-4476-9831-385ac17e3eb7 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.021851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14732a-d796-4476-9831-385ac17e3eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.022354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd14732a-d796-4476-9831-385ac17e3eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.036259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1098/2219] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:14.038167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d263734-4854-4b19-b635-3ea2dcb344f4 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.052267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d263734-4854-4b19-b635-3ea2dcb344f4 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.054817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cd14732a-d796-4476-9831-385ac17e3eb7 None None] MFA Rules not processed for user `adb792e7246d42dface4aa52899a0ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:14.056029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cd14732a-d796-4476-9831-385ac17e3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:14.058333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d49bba-a169-47bb-b0c8-38d8ba2e0ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.059761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d49bba-a169-47bb-b0c8-38d8ba2e0ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.060296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d49bba-a169-47bb-b0c8-38d8ba2e0ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.061314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d49bba-a169-47bb-b0c8-38d8ba2e0ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.062199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d49bba-a169-47bb-b0c8-38d8ba2e0ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.064784 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d49bba-a169-47bb-b0c8-38d8ba2e0ff7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.155296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1122/2220] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:14.156397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14732a-d796-4476-9831-385ac17e3eb7 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.157756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cd14732a-d796-4476-9831-385ac17e3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:14.172208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd14732a-d796-4476-9831-385ac17e3eb7 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.177243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.177857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.178140 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.178416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.179073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.181003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.197240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1099/2221] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:14 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:14.199911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3339505-f4a9-44d2-9993-83d18190589e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.200555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3339505-f4a9-44d2-9993-83d18190589e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.201048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3339505-f4a9-44d2-9993-83d18190589e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.219471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:14.223386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c3339505-f4a9-44d2-9993-83d18190589e None None] MFA Rules not processed for user `adb792e7246d42dface4aa52899a0ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:14.225020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c3339505-f4a9-44d2-9993-83d18190589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:14.285677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1100/2222] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:14.286658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3339505-f4a9-44d2-9993-83d18190589e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.316738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1123/2223] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:14 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:14.316738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.317342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:14.359413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3339505-f4a9-44d2-9993-83d18190589e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.403997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda3e1f-2981-472c-998d-c784c1c6f430 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.410134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c0e68-ada0-459d-87a5-319da7708028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.410794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c0e68-ada0-459d-87a5-319da7708028 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.411142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c0e68-ada0-459d-87a5-319da7708028 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.411417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c0e68-ada0-459d-87a5-319da7708028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.419861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.421755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.422046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.422346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.423193 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.426608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.483799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c0e68-ada0-459d-87a5-319da7708028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.483799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c0e68-ada0-459d-87a5-319da7708028 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.493486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1124/2224] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:14.494046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c0e68-ada0-459d-87a5-319da7708028 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.506043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c0e68-ada0-459d-87a5-319da7708028 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.517816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b75038-d164-443e-b720-11f4579a3bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.517816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b75038-d164-443e-b720-11f4579a3bca admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.517816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b75038-d164-443e-b720-11f4579a3bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.517816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b75038-d164-443e-b720-11f4579a3bca admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.518230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b75038-d164-443e-b720-11f4579a3bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.518230 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b75038-d164-443e-b720-11f4579a3bca admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.579064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1101/2225] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:14 2022] POST /identity/v3/users => generated 287 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:14.580019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.582251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:14.597496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a606d-5c29-4976-8a95-23e90d3be274 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.604931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.605810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.606346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.606684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] PATH_INFO: `/v3/projects/42ce13e2223a4a3297b4eb0892afaefe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.608933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ce13e2223a4a3297b4eb0892afaefe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.611235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.673340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1125/2226] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:14 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:14.674700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b75038-d164-443e-b720-11f4579a3bca admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.694240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b75038-d164-443e-b720-11f4579a3bca admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.704075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd51368-96f1-4b42-b277-6234aa1411ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.704889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51368-96f1-4b42-b277-6234aa1411ab admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.705251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51368-96f1-4b42-b277-6234aa1411ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.709490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd51368-96f1-4b42-b277-6234aa1411ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.709490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd51368-96f1-4b42-b277-6234aa1411ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.710448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd51368-96f1-4b42-b277-6234aa1411ab admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.722795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1126/2227] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:14.723814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd51368-96f1-4b42-b277-6234aa1411ab admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.735154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd51368-96f1-4b42-b277-6234aa1411ab admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.741699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06687e-9c1e-448e-b787-02f2637032d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.742389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06687e-9c1e-448e-b787-02f2637032d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.742699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06687e-9c1e-448e-b787-02f2637032d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.742981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06687e-9c1e-448e-b787-02f2637032d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.744760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06687e-9c1e-448e-b787-02f2637032d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.748902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06687e-9c1e-448e-b787-02f2637032d8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.763202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1127/2228] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:14.765996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06687e-9c1e-448e-b787-02f2637032d8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.784783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06687e-9c1e-448e-b787-02f2637032d8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.834822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ce13e2223a4a3297b4eb0892afaefe', 'request_id': 'req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:14.855121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ce13e2223a4a3297b4eb0892afaefe', 'request_id': 'req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:14.866529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ce13e2223a4a3297b4eb0892afaefe', 'request_id': 'req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:14.921578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb149bc-9fde-47a8-a589-d91c88acf05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.922494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb149bc-9fde-47a8-a589-d91c88acf05a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.922969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb149bc-9fde-47a8-a589-d91c88acf05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.923277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb149bc-9fde-47a8-a589-d91c88acf05a admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.923984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb149bc-9fde-47a8-a589-d91c88acf05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:14.927222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb149bc-9fde-47a8-a589-d91c88acf05a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:14.950951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1128/2229] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:14.957267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb149bc-9fde-47a8-a589-d91c88acf05a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.974764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb149bc-9fde-47a8-a589-d91c88acf05a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:14.991418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] The token cache is being invalidate because project 42ce13e2223a4a3297b4eb0892afaefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:14.991788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:14.992655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:14.994077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62279-283b-4527-bf8c-bb29cec9aa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:14.994424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1102/2230] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:14 2022] DELETE /identity/v3/projects/42ce13e2223a4a3297b4eb0892afaefe => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:14.995242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:14.995755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62279-283b-4527-bf8c-bb29cec9aa6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:14.996570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62279-283b-4527-bf8c-bb29cec9aa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:14.997206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62279-283b-4527-bf8c-bb29cec9aa6b admin admin] PATH_INFO: `/v3/projects/113dae4e9c7f47b29f7da269b1a6c642/users/b6f164e7a0e146bcbada0391a542f1fe/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:14.999583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.017300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db93cf0-0ed9-46f3-a354-f7a509f2f14c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:15.018039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b62279-283b-4527-bf8c-bb29cec9aa6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=113dae4e9c7f47b29f7da269b1a6c642, user_id=b6f164e7a0e146bcbada0391a542f1fe, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.021514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b62279-283b-4527-bf8c-bb29cec9aa6b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.023830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:15.024851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.025300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.025721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] PATH_INFO: `/v3/projects/e9fac55b82fc47fc94aae510737d4369/users/554f6d5f3e7f420fb2212bc5185f5b20/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.068686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1129/2231] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:14 2022] PUT /identity/v3/projects/113dae4e9c7f47b29f7da269b1a6c642/users/b6f164e7a0e146bcbada0391a542f1fe/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:15.070527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9e9a4-245c-4b43-a1e4-4bda9662faff None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.070952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9e9a4-245c-4b43-a1e4-4bda9662faff None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.071412 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9fac55b82fc47fc94aae510737d4369, user_id=554f6d5f3e7f420fb2212bc5185f5b20, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.072371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9e9a4-245c-4b43-a1e4-4bda9662faff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.088327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.109228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b5a9e9a4-245c-4b43-a1e4-4bda9662faff None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:15.110700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b5a9e9a4-245c-4b43-a1e4-4bda9662faff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.155810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1103/2232] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:14 2022] PUT /identity/v3/projects/e9fac55b82fc47fc94aae510737d4369/users/554f6d5f3e7f420fb2212bc5185f5b20/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:15.157208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:15.159177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.189533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1130/2233] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:15.198194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803204e4-a7b5-4d7d-ae8c-12a7452c07ea None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.203460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803204e4-a7b5-4d7d-ae8c-12a7452c07ea None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.203460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803204e4-a7b5-4d7d-ae8c-12a7452c07ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.231065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-803204e4-a7b5-4d7d-ae8c-12a7452c07ea None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:15.232834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-803204e4-a7b5-4d7d-ae8c-12a7452c07ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.236863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d45477-f426-4ee6-b8fd-b3b4d03877a0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:15.246374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:15.249967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1131/2234] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:15.251914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.252453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d4acc-efa9-4690-8642-25b9d935dd00 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.252865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d4acc-efa9-4690-8642-25b9d935dd00 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.253041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d4acc-efa9-4690-8642-25b9d935dd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.254404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.254914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.255805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.257733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.271562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1104/2235] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:15.273302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:15.279409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.282057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8b3d4acc-efa9-4690-8642-25b9d935dd00 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:15.283407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8b3d4acc-efa9-4690-8642-25b9d935dd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.301033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1132/2236] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:15.301033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d4acc-efa9-4690-8642-25b9d935dd00 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:15.312050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d4acc-efa9-4690-8642-25b9d935dd00 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:15.318185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd00984-934b-4b10-81e3-186e5ece84e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:15.319292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd00984-934b-4b10-81e3-186e5ece84e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.319898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd00984-934b-4b10-81e3-186e5ece84e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.320431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd00984-934b-4b10-81e3-186e5ece84e6 admin admin] PATH_INFO: `/v3/projects/113dae4e9c7f47b29f7da269b1a6c642/users/b6f164e7a0e146bcbada0391a542f1fe/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.323175 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd00984-934b-4b10-81e3-186e5ece84e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=113dae4e9c7f47b29f7da269b1a6c642, user_id=b6f164e7a0e146bcbada0391a542f1fe, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.327107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd00984-934b-4b10-81e3-186e5ece84e6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.344360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1133/2237] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:15 2022] PUT /identity/v3/projects/113dae4e9c7f47b29f7da269b1a6c642/users/b6f164e7a0e146bcbada0391a542f1fe/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:15.346045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd00984-934b-4b10-81e3-186e5ece84e6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:15.390701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bf6f0-16f7-4d2f-9108-fd87795b2d3a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:15.441052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd00984-934b-4b10-81e3-186e5ece84e6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:15.447997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac8982-d749-40a6-b1d8-6babcdf2dc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:15.448687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa712313-6dc1-46fc-9a50-421daf5ecb23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:15.449202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712313-6dc1-46fc-9a50-421daf5ecb23 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.449486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712313-6dc1-46fc-9a50-421daf5ecb23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.449761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa712313-6dc1-46fc-9a50-421daf5ecb23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.450256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8982-d749-40a6-b1d8-6babcdf2dc69 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.450655 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8982-d749-40a6-b1d8-6babcdf2dc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.451052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac8982-d749-40a6-b1d8-6babcdf2dc69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.452870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac8982-d749-40a6-b1d8-6babcdf2dc69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:40:15.453223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac8982-d749-40a6-b1d8-6babcdf2dc69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.454855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac8982-d749-40a6-b1d8-6babcdf2dc69 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.461200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa712313-6dc1-46fc-9a50-421daf5ecb23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.462769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa712313-6dc1-46fc-9a50-421daf5ecb23 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.474465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1134/2238] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:40:15.475515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1105/2239] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:15.476458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa712313-6dc1-46fc-9a50-421daf5ecb23 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:15.478015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a0159-b686-44eb-834a-fb27864d57be None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.478500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a0159-b686-44eb-834a-fb27864d57be None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.478947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a0159-b686-44eb-834a-fb27864d57be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.491978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa712313-6dc1-46fc-9a50-421daf5ecb23 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:15.502490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da7c0a-3ec4-4a8a-ba9d-32a812659040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:15.503258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da7c0a-3ec4-4a8a-ba9d-32a812659040 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.503673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da7c0a-3ec4-4a8a-ba9d-32a812659040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.503841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da7c0a-3ec4-4a8a-ba9d-32a812659040 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.504870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da7c0a-3ec4-4a8a-ba9d-32a812659040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.506480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da7c0a-3ec4-4a8a-ba9d-32a812659040 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.521209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cf4a0159-b686-44eb-834a-fb27864d57be None None] MFA Rules not processed for user `b6f164e7a0e146bcbada0391a542f1fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:15.522508 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cf4a0159-b686-44eb-834a-fb27864d57be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.600890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1135/2240] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:15.615309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a0159-b686-44eb-834a-fb27864d57be None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:15.617386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cf4a0159-b686-44eb-834a-fb27864d57be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.677393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1106/2241] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:15 2022] POST /identity/v3/projects => generated 399 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:15.680690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da7c0a-3ec4-4a8a-ba9d-32a812659040 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:15.752940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a0159-b686-44eb-834a-fb27864d57be None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:15.824434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da7c0a-3ec4-4a8a-ba9d-32a812659040 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:15.826445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af2339-7d55-461a-9007-2e727cf2b552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:15.827770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af2339-7d55-461a-9007-2e727cf2b552 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.828058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af2339-7d55-461a-9007-2e727cf2b552 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.829229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af2339-7d55-461a-9007-2e727cf2b552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.831161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-05af2339-7d55-461a-9007-2e727cf2b552 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.834532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96720dc-9482-4eb4-a355-b11ec02c05eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:15.843089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96720dc-9482-4eb4-a355-b11ec02c05eb admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.843089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96720dc-9482-4eb4-a355-b11ec02c05eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.843089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96720dc-9482-4eb4-a355-b11ec02c05eb admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.843089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96720dc-9482-4eb4-a355-b11ec02c05eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.843089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96720dc-9482-4eb4-a355-b11ec02c05eb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.870608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:15.910920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af2339-7d55-461a-9007-2e727cf2b552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:15.912745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05af2339-7d55-461a-9007-2e727cf2b552 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:15.932863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1136/2242] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:40:15.936122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba1d71-cd62-47fa-bb76-bf995656570e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:15.936454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba1d71-cd62-47fa-bb76-bf995656570e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:15.936923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba1d71-cd62-47fa-bb76-bf995656570e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:15.958065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-96ba1d71-cd62-47fa-bb76-bf995656570e None None] MFA Rules not processed for user `adb792e7246d42dface4aa52899a0ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:15.959309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-96ba1d71-cd62-47fa-bb76-bf995656570e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:15.968148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1107/2243] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:15 2022] POST /identity/v3/users => generated 300 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:15.969521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96720dc-9482-4eb4-a355-b11ec02c05eb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:15.987414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96720dc-9482-4eb4-a355-b11ec02c05eb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:16.005068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9449c1-ef2f-408f-a4b4-7ac6af35c6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:16.005600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9449c1-ef2f-408f-a4b4-7ac6af35c6ff admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:16.005885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9449c1-ef2f-408f-a4b4-7ac6af35c6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:16.006166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9449c1-ef2f-408f-a4b4-7ac6af35c6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:16.006838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9449c1-ef2f-408f-a4b4-7ac6af35c6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:16.008664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9449c1-ef2f-408f-a4b4-7ac6af35c6ff admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:16.027898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1108/2244] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:16.032415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9449c1-ef2f-408f-a4b4-7ac6af35c6ff admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:16.046621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9449c1-ef2f-408f-a4b4-7ac6af35c6ff admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:16.049192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1137/2245] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:16.053201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12af036-d1c9-4133-867c-ec1f7016d85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:16.054000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12af036-d1c9-4133-867c-ec1f7016d85d admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:16.065270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12af036-d1c9-4133-867c-ec1f7016d85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:16.065270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12af036-d1c9-4133-867c-ec1f7016d85d admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:16.065270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12af036-d1c9-4133-867c-ec1f7016d85d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:16.065270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12af036-d1c9-4133-867c-ec1f7016d85d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:16.071059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1109/2246] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:40:16.076896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba1d71-cd62-47fa-bb76-bf995656570e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:16.127431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba1d71-cd62-47fa-bb76-bf995656570e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:16.149000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941db0aa-a15d-4acc-992d-95682a155801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:16.153348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941db0aa-a15d-4acc-992d-95682a155801 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:16.153684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941db0aa-a15d-4acc-992d-95682a155801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:16.155659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941db0aa-a15d-4acc-992d-95682a155801 admin admin] PATH_INFO: `/v3/projects/65869b4648154c4bbb7d0d86de37de7b/users/ea21b6783f0943b4ba77f9e2bf4b5955/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:16.206676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941db0aa-a15d-4acc-992d-95682a155801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65869b4648154c4bbb7d0d86de37de7b, user_id=ea21b6783f0943b4ba77f9e2bf4b5955, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:16.209651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941db0aa-a15d-4acc-992d-95682a155801 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:16.237130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528bc7-2369-4381-afe7-9ed707b1b4f3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:16.237873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528bc7-2369-4381-afe7-9ed707b1b4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:16.238327 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe528bc7-2369-4381-afe7-9ed707b1b4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:16.275254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fe528bc7-2369-4381-afe7-9ed707b1b4f3 None None] MFA Rules not processed for user `e9c7ecfdb9904095b122bb8fd82121fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:16.277078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fe528bc7-2369-4381-afe7-9ed707b1b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:16.324299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1138/2247] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:16 2022] PUT /identity/v3/projects/65869b4648154c4bbb7d0d86de37de7b/users/ea21b6783f0943b4ba77f9e2bf4b5955/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:16.336490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb941c-4377-4dd8-9ffc-4798ab925f11 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:16.336490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb941c-4377-4dd8-9ffc-4798ab925f11 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:16.336797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb941c-4377-4dd8-9ffc-4798ab925f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:16.382095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1110/2248] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:16.382309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-59eb941c-4377-4dd8-9ffc-4798ab925f11 None None] MFA Rules not processed for user `ea21b6783f0943b4ba77f9e2bf4b5955`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:16.382757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-59eb941c-4377-4dd8-9ffc-4798ab925f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:16.509976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1139/2249] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:40:16.519497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe528bc7-2369-4381-afe7-9ed707b1b4f3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:16.523957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fe528bc7-2369-4381-afe7-9ed707b1b4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:16.554649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe528bc7-2369-4381-afe7-9ed707b1b4f3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:16.642449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003ee17-048d-4d35-84dd-1ad6071d474c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:16.643096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003ee17-048d-4d35-84dd-1ad6071d474c service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:16.643480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003ee17-048d-4d35-84dd-1ad6071d474c service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:16.643905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003ee17-048d-4d35-84dd-1ad6071d474c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:16.655013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003ee17-048d-4d35-84dd-1ad6071d474c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:16.656921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003ee17-048d-4d35-84dd-1ad6071d474c service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:16.679716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1111/2250] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:16.710595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:18.159114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6127f-a261-4b70-b9d6-38788a688eed None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:18.159703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6127f-a261-4b70-b9d6-38788a688eed None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:18.159872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6127f-a261-4b70-b9d6-38788a688eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:18.241901 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d2b6127f-a261-4b70-b9d6-38788a688eed None None] MFA Rules not processed for user `554f6d5f3e7f420fb2212bc5185f5b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:18.243611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d2b6127f-a261-4b70-b9d6-38788a688eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:18.381974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1140/2251] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:18.386490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003ee17-048d-4d35-84dd-1ad6071d474c service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:18.484629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6127f-a261-4b70-b9d6-38788a688eed None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:18.495581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6127f-a261-4b70-b9d6-38788a688eed None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:18.500391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:18.501201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:18.501489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:18.501762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:18.503585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:18.569090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003ee17-048d-4d35-84dd-1ad6071d474c service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:18.587173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2464a1-1f2f-4486-9bf2-eda5e7ce7148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:18.587897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2464a1-1f2f-4486-9bf2-eda5e7ce7148 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:18.588238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2464a1-1f2f-4486-9bf2-eda5e7ce7148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:18.588556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2464a1-1f2f-4486-9bf2-eda5e7ce7148 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:18.589153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2464a1-1f2f-4486-9bf2-eda5e7ce7148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:18.591133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2464a1-1f2f-4486-9bf2-eda5e7ce7148 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:18.670022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:18.688896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:18.706310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:18.713107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1141/2252] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:18.758521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1112/2253] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:18 2022] POST /identity/v3/projects => generated 371 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:18.759903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:18.793944 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd110b-fc2c-4a3d-bb1c-1ce044e2ee2b service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:18.903747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2bfc7-8379-42ec-a51c-9b1fa4390336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:18.905185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2bfc7-8379-42ec-a51c-9b1fa4390336 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:18.905324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2bfc7-8379-42ec-a51c-9b1fa4390336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:18.905591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2bfc7-8379-42ec-a51c-9b1fa4390336 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:18.906265 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2bfc7-8379-42ec-a51c-9b1fa4390336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:18.908512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2bfc7-8379-42ec-a51c-9b1fa4390336 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:19.058236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1142/2254] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:18 2022] POST /identity/v3/users => generated 292 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:40:19.065172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2464a1-1f2f-4486-9bf2-eda5e7ce7148 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:19.097249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2464a1-1f2f-4486-9bf2-eda5e7ce7148 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:19.108981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b95df-533c-4a82-8c53-060580a7d397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:19.109928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b95df-533c-4a82-8c53-060580a7d397 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:19.110278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b95df-533c-4a82-8c53-060580a7d397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:19.110739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b95df-533c-4a82-8c53-060580a7d397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:19.111981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b95df-533c-4a82-8c53-060580a7d397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:19.114955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b95df-533c-4a82-8c53-060580a7d397 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:19.136834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1113/2255] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:19.141319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2bfc7-8379-42ec-a51c-9b1fa4390336 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:19.156948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2bfc7-8379-42ec-a51c-9b1fa4390336 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:19.163307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566d062-4555-47c4-8d9e-28192afcc5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:19.164084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566d062-4555-47c4-8d9e-28192afcc5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:19.164407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566d062-4555-47c4-8d9e-28192afcc5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:19.164844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566d062-4555-47c4-8d9e-28192afcc5b7 admin admin] PATH_INFO: `/v3/projects/3688a3a1a4274476a01dca2c895e2146/users/dc91904a779b4288985d17f5bf79fbb0/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:19.187578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566d062-4555-47c4-8d9e-28192afcc5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3688a3a1a4274476a01dca2c895e2146, user_id=dc91904a779b4288985d17f5bf79fbb0, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:19.190519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566d062-4555-47c4-8d9e-28192afcc5b7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:19.306427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1143/2256] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:19 2022] PUT /identity/v3/projects/3688a3a1a4274476a01dca2c895e2146/users/dc91904a779b4288985d17f5bf79fbb0/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:19.312507 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b95df-533c-4a82-8c53-060580a7d397 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:19.328693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b95df-533c-4a82-8c53-060580a7d397 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:19.413359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db91f-fc97-4464-a970-d7b7c079f0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:19.414053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db91f-fc97-4464-a970-d7b7c079f0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:19.414243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db91f-fc97-4464-a970-d7b7c079f0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:19.414488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026db91f-fc97-4464-a970-d7b7c079f0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:19.415162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026db91f-fc97-4464-a970-d7b7c079f0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:19.417128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026db91f-fc97-4464-a970-d7b7c079f0b8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:19.427547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1114/2257] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:19.431966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566d062-4555-47c4-8d9e-28192afcc5b7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:19.443555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566d062-4555-47c4-8d9e-28192afcc5b7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:19.448379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7f85-f78c-4715-8872-1b319332a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:19.449023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7f85-f78c-4715-8872-1b319332a149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:19.449300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7f85-f78c-4715-8872-1b319332a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:19.449576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7f85-f78c-4715-8872-1b319332a149 admin admin] PATH_INFO: `/v3/projects/3688a3a1a4274476a01dca2c895e2146/users/dc91904a779b4288985d17f5bf79fbb0/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:19.452258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a7f85-f78c-4715-8872-1b319332a149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3688a3a1a4274476a01dca2c895e2146, user_id=dc91904a779b4288985d17f5bf79fbb0, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:19.455245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a7f85-f78c-4715-8872-1b319332a149 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:19.466531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1144/2258] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:19 2022] PUT /identity/v3/projects/3688a3a1a4274476a01dca2c895e2146/users/dc91904a779b4288985d17f5bf79fbb0/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:20.409433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:21.178966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db91f-fc97-4464-a970-d7b7c079f0b8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:21.249757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026db91f-fc97-4464-a970-d7b7c079f0b8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:21.367750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c906c49-cd2e-4293-bdd0-6523c0a1c769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:21.368528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906c49-cd2e-4293-bdd0-6523c0a1c769 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:21.369061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906c49-cd2e-4293-bdd0-6523c0a1c769 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:21.369281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c906c49-cd2e-4293-bdd0-6523c0a1c769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:21.448957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c906c49-cd2e-4293-bdd0-6523c0a1c769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:21.450605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c906c49-cd2e-4293-bdd0-6523c0a1c769 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:21.462236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1115/2259] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:21.641127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0865fd-71f1-4002-aed7-31b39f9468e6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:21.641719 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0865fd-71f1-4002-aed7-31b39f9468e6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:21.641925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0865fd-71f1-4002-aed7-31b39f9468e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:21.676135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-4e0865fd-71f1-4002-aed7-31b39f9468e6 None None] MFA Rules not processed for user `dc91904a779b4288985d17f5bf79fbb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:21.677759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4e0865fd-71f1-4002-aed7-31b39f9468e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:21.784766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1145/2260] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:21.803124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c906c49-cd2e-4293-bdd0-6523c0a1c769 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:21.812382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c906c49-cd2e-4293-bdd0-6523c0a1c769 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:21.816056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234579e8-949b-4839-a3ed-d3480dcd8e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:21.816687 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234579e8-949b-4839-a3ed-d3480dcd8e24 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:21.817039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234579e8-949b-4839-a3ed-d3480dcd8e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:21.817317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234579e8-949b-4839-a3ed-d3480dcd8e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:21.881426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234579e8-949b-4839-a3ed-d3480dcd8e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:21.882948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234579e8-949b-4839-a3ed-d3480dcd8e24 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:21.893354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1116/2261] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:23.063666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:23.895840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:24.792700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0865fd-71f1-4002-aed7-31b39f9468e6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:24.805084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0865fd-71f1-4002-aed7-31b39f9468e6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:24.809980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67482c1e-67ce-4d24-bdb8-0210e9c31020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:24.811517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67482c1e-67ce-4d24-bdb8-0210e9c31020 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:24.813318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67482c1e-67ce-4d24-bdb8-0210e9c31020 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:24.813318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67482c1e-67ce-4d24-bdb8-0210e9c31020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:24.828212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67482c1e-67ce-4d24-bdb8-0210e9c31020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:24.830151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67482c1e-67ce-4d24-bdb8-0210e9c31020 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:24.847849 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1146/2262] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:26.288814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:27.486243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234579e8-949b-4839-a3ed-d3480dcd8e24 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:27.552872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:27.615895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234579e8-949b-4839-a3ed-d3480dcd8e24 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:27.623730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:27.625175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:27.625983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:27.626459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] PATH_INFO: `/v3/users/ea21b6783f0943b4ba77f9e2bf4b5955` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:27.633084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea21b6783f0943b4ba77f9e2bf4b5955)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:27.636457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:27.948873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea21b6783f0943b4ba77f9e2bf4b5955'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:27.949726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea21b6783f0943b4ba77f9e2bf4b5955'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:27.965637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea21b6783f0943b4ba77f9e2bf4b5955'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:27.966396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea21b6783f0943b4ba77f9e2bf4b5955'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:28.046254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea21b6783f0943b4ba77f9e2bf4b5955'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:28.079843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1117/2263] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:27 2022] DELETE /identity/v3/users/ea21b6783f0943b4ba77f9e2bf4b5955 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:28.087381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d356e0-7136-4928-8364-eb87b356f016 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:28.088462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d356e0-7136-4928-8364-eb87b356f016 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:28.089701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d356e0-7136-4928-8364-eb87b356f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:28.120830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-02d356e0-7136-4928-8364-eb87b356f016 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:28.122685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-02d356e0-7136-4928-8364-eb87b356f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:28.255280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1147/2264] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:28.266254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:28.268377 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:28.363056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b224286-1b78-4a6a-92ff-2865066b5b0b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:28.367167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:28.368069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:28.368423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:28.368725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:28.380720 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:28.381971 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:28.393322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1118/2265] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:29.275400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d356e0-7136-4928-8364-eb87b356f016 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:29.277882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-02d356e0-7136-4928-8364-eb87b356f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:29.301324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d356e0-7136-4928-8364-eb87b356f016 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:29.312544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:29.313480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:29.314866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:29.316052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] PATH_INFO: `/v3/projects/65869b4648154c4bbb7d0d86de37de7b` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:29.321095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65869b4648154c4bbb7d0d86de37de7b)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:29.328005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:29.421921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:29.423428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:29.439329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2f25a-7279-42b5-8d36-f4dd2150a2a0 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:29.444318 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:29.444960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:29.445247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:29.445522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] PATH_INFO: `/v3/users/d56decda97274523955b0c38d1cae646` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:29.446686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56decda97274523955b0c38d1cae646)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:29.449001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:29.543621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65869b4648154c4bbb7d0d86de37de7b', 'request_id': 'req-3876f929-3b3f-4919-9915-cfa7ad97b92a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.553258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65869b4648154c4bbb7d0d86de37de7b', 'request_id': 'req-3876f929-3b3f-4919-9915-cfa7ad97b92a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.655184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65869b4648154c4bbb7d0d86de37de7b', 'request_id': 'req-3876f929-3b3f-4919-9915-cfa7ad97b92a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.671775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] The token cache is being invalidate because project 65869b4648154c4bbb7d0d86de37de7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:29.672167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.672972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3876f929-3b3f-4919-9915-cfa7ad97b92a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.675015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1148/2266] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:29 2022] DELETE /identity/v3/projects/65869b4648154c4bbb7d0d86de37de7b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:29.677323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f76e-f4ae-490a-84a2-7755947406fa None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:29.679180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f76e-f4ae-490a-84a2-7755947406fa None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:29.679180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649f76e-f4ae-490a-84a2-7755947406fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:29.705829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8649f76e-f4ae-490a-84a2-7755947406fa None None] MFA Rules not processed for user `ea9d4eafe59c4c8180eece8ec9283e40`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:29.707145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8649f76e-f4ae-490a-84a2-7755947406fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:29.721614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56decda97274523955b0c38d1cae646'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.722341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56decda97274523955b0c38d1cae646'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.744049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:29.749144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56decda97274523955b0c38d1cae646'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.749970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56decda97274523955b0c38d1cae646'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.831582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1149/2267] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:29.831582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae0d31-98f2-4469-b887-f98f1a39795d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:29.831582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae0d31-98f2-4469-b887-f98f1a39795d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:29.831582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae0d31-98f2-4469-b887-f98f1a39795d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:29.854454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5fae0d31-98f2-4469-b887-f98f1a39795d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:29.855557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5fae0d31-98f2-4469-b887-f98f1a39795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:29.861475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-fadf5ba4-bad5-4f73-9677-56044ac2a08c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56decda97274523955b0c38d1cae646'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:29.896052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1119/2268] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:29 2022] DELETE /identity/v3/users/d56decda97274523955b0c38d1cae646 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:40:29.897612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf70f-1749-4779-8e59-e6badea44cc3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:29.897906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf70f-1749-4779-8e59-e6badea44cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:29.898256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf70f-1749-4779-8e59-e6badea44cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:29.922646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f79cf70f-1749-4779-8e59-e6badea44cc3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:29.928821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f79cf70f-1749-4779-8e59-e6badea44cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:29.955138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1150/2269] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:29.965481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71977698-82ac-4300-9447-03c7291c6ff9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:29.967900 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71977698-82ac-4300-9447-03c7291c6ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:29.968694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71977698-82ac-4300-9447-03c7291c6ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:29.993169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-71977698-82ac-4300-9447-03c7291c6ff9 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:29.994832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-71977698-82ac-4300-9447-03c7291c6ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:30.006655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1120/2270] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:30.017954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cf70f-1749-4779-8e59-e6badea44cc3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:30.019869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f79cf70f-1749-4779-8e59-e6badea44cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:30.137879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1151/2271] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:30.141200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71977698-82ac-4300-9447-03c7291c6ff9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:30.150990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71977698-82ac-4300-9447-03c7291c6ff9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:30.158560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cf70f-1749-4779-8e59-e6badea44cc3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:30.165488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:30.166117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:30.166399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:30.166671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:30.167338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:40:30.167641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:30.169401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:30.174769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dee61-1607-45e4-86ef-c76aa6fcc192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:30.176499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dee61-1607-45e4-86ef-c76aa6fcc192 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:30.177417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dee61-1607-45e4-86ef-c76aa6fcc192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:30.178102 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dee61-1607-45e4-86ef-c76aa6fcc192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:30.188589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1152/2272] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:40:30.194075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:30.205019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dee61-1607-45e4-86ef-c76aa6fcc192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:30.211041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ccca2-248f-48a1-96ac-8e8ba13b4c40 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:30.211188 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dee61-1607-45e4-86ef-c76aa6fcc192 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:30.213466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d9e21-8535-4b5c-abf0-4f334c2fda4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:30.214176 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d9e21-8535-4b5c-abf0-4f334c2fda4f admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:30.214621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d9e21-8535-4b5c-abf0-4f334c2fda4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:30.214971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d9e21-8535-4b5c-abf0-4f334c2fda4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:30.215840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d9e21-8535-4b5c-abf0-4f334c2fda4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:30.218129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d9e21-8535-4b5c-abf0-4f334c2fda4f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:30.235622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1121/2273] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:30.326458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1153/2274] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:30 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:30.330932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dee61-1607-45e4-86ef-c76aa6fcc192 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:30.342627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dee61-1607-45e4-86ef-c76aa6fcc192 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:30.417146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05679199-3fd8-44db-978a-468a5712c041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:30.417745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05679199-3fd8-44db-978a-468a5712c041 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:30.418023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05679199-3fd8-44db-978a-468a5712c041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:30.418781 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05679199-3fd8-44db-978a-468a5712c041 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:30.418962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05679199-3fd8-44db-978a-468a5712c041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:30.421915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05679199-3fd8-44db-978a-468a5712c041 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:30.546264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1122/2275] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:30 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:40:30.550572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d9e21-8535-4b5c-abf0-4f334c2fda4f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:30.579549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d9e21-8535-4b5c-abf0-4f334c2fda4f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:30.585576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a43904-b0fd-4c92-9961-d1caaa0c489a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:30.586207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a43904-b0fd-4c92-9961-d1caaa0c489a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:30.586463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a43904-b0fd-4c92-9961-d1caaa0c489a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:30.586735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a43904-b0fd-4c92-9961-d1caaa0c489a admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:30.587391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a43904-b0fd-4c92-9961-d1caaa0c489a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:30.589159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a43904-b0fd-4c92-9961-d1caaa0c489a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:30.608800 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1154/2276] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:30.610144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05679199-3fd8-44db-978a-468a5712c041 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:30.625976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05679199-3fd8-44db-978a-468a5712c041 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:30.632789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffc415-0bb7-4123-85d8-988919f1a32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:30.633780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffc415-0bb7-4123-85d8-988919f1a32c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:30.633780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffc415-0bb7-4123-85d8-988919f1a32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:30.633976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffc415-0bb7-4123-85d8-988919f1a32c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:30.634758 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffc415-0bb7-4123-85d8-988919f1a32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:30.636411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffc415-0bb7-4123-85d8-988919f1a32c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:30.652532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1123/2277] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:30.655223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a43904-b0fd-4c92-9961-d1caaa0c489a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:30.665941 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a43904-b0fd-4c92-9961-d1caaa0c489a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:30.674733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:30.675711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:30.675988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:30.676339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] PATH_INFO: `/v3/projects/6bcafe839dec4fcc95a4c720463483ba/users/2730741d853a4cfcab1bbc990cf42611/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:30.692581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcafe839dec4fcc95a4c720463483ba, user_id=2730741d853a4cfcab1bbc990cf42611, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:30.695459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:30.805736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1155/2278] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:30 2022] PUT /identity/v3/projects/6bcafe839dec4fcc95a4c720463483ba/users/2730741d853a4cfcab1bbc990cf42611/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:30.812703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a89d8-cb62-46a5-8a92-49fd10c51a7b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:30.813441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a89d8-cb62-46a5-8a92-49fd10c51a7b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:30.814128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a89d8-cb62-46a5-8a92-49fd10c51a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:30.854921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-430a89d8-cb62-46a5-8a92-49fd10c51a7b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:30.858008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-430a89d8-cb62-46a5-8a92-49fd10c51a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:30.873502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:30.876420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:30.986972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1124/2279] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:31.001143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a89d8-cb62-46a5-8a92-49fd10c51a7b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:31.003293 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-430a89d8-cb62-46a5-8a92-49fd10c51a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:31.022320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac195c-7fba-4c72-9032-83a8d8815b82 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:31.031264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:31.034202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:31.034202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:31.034202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] PATH_INFO: `/v3/projects/609ef41523c24dc09131a8a8ac829805` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:31.034202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] RBAC: Authorizing `identity:delete_project(project_id=609ef41523c24dc09131a8a8ac829805)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:31.036037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:31.073035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a89d8-cb62-46a5-8a92-49fd10c51a7b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:31.076848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d717d6-85df-4bb4-9ac5-a031ed7744e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:31.077457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d717d6-85df-4bb4-9ac5-a031ed7744e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:31.077739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d717d6-85df-4bb4-9ac5-a031ed7744e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:31.078012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d717d6-85df-4bb4-9ac5-a031ed7744e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:31.087755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d717d6-85df-4bb4-9ac5-a031ed7744e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:31.089355 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d717d6-85df-4bb4-9ac5-a031ed7744e9 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:31.101823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1125/2280] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:31.207293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609ef41523c24dc09131a8a8ac829805', 'request_id': 'req-3ccadf2b-76c0-4327-a60f-d7a8630489da'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.214425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609ef41523c24dc09131a8a8ac829805', 'request_id': 'req-3ccadf2b-76c0-4327-a60f-d7a8630489da'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.306214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609ef41523c24dc09131a8a8ac829805', 'request_id': 'req-3ccadf2b-76c0-4327-a60f-d7a8630489da'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.359658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] The token cache is being invalidate because project 609ef41523c24dc09131a8a8ac829805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:31.360325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.362189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.366157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1156/2281] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:30 2022] DELETE /identity/v3/projects/609ef41523c24dc09131a8a8ac829805 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:31.370867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d717d6-85df-4bb4-9ac5-a031ed7744e9 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:31.387315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d717d6-85df-4bb4-9ac5-a031ed7744e9 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:31.450384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:31.489552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:31.490365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:31.491827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:31.492740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] PATH_INFO: `/v3/users/7fc5b0225c914dc9bc3c64ffc53f1324` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:31.495946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc5b0225c914dc9bc3c64ffc53f1324)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:31.502333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:31.780782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc5b0225c914dc9bc3c64ffc53f1324'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.782969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc5b0225c914dc9bc3c64ffc53f1324'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.792372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc5b0225c914dc9bc3c64ffc53f1324'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.793221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc5b0225c914dc9bc3c64ffc53f1324'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.883745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-8961394e-3aef-4c8d-b92f-e85a860a4256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc5b0225c914dc9bc3c64ffc53f1324'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:31.914351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1126/2282] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:31 2022] DELETE /identity/v3/users/7fc5b0225c914dc9bc3c64ffc53f1324 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:32.879663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:32.882351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:32.906749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccadf2b-76c0-4327-a60f-d7a8630489da admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:32.990393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fece293-cfba-4c7c-bb53-b68879771b26 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:32.991041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fece293-cfba-4c7c-bb53-b68879771b26 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:32.991248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fece293-cfba-4c7c-bb53-b68879771b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:33.018589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1fece293-cfba-4c7c-bb53-b68879771b26 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:33.020235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1fece293-cfba-4c7c-bb53-b68879771b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:33.094029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:33.095050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:33.095343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:33.095747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] PATH_INFO: `/v3/projects/09be5b26299442109a350b9d82b94887` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:33.098173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09be5b26299442109a350b9d82b94887)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:33.100461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:33.180378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1127/2283] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:40:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:40:33.212834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fece293-cfba-4c7c-bb53-b68879771b26 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:33.214425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1fece293-cfba-4c7c-bb53-b68879771b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:33.261423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:33.284114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09be5b26299442109a350b9d82b94887', 'request_id': 'req-bc188837-ae24-4fc6-97ec-4984ef53c560'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:33.298568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09be5b26299442109a350b9d82b94887', 'request_id': 'req-bc188837-ae24-4fc6-97ec-4984ef53c560'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:33.301087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fece293-cfba-4c7c-bb53-b68879771b26 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:33.368043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:33.368858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:33.369142 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:33.369418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:33.386264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09be5b26299442109a350b9d82b94887', 'request_id': 'req-bc188837-ae24-4fc6-97ec-4984ef53c560'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:33.407267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] The token cache is being invalidate because project 09be5b26299442109a350b9d82b94887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:33.407566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:33.408343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:33.413038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1157/2284] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:32 2022] DELETE /identity/v3/projects/09be5b26299442109a350b9d82b94887 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:33.419001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:33.422181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:33.508991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:33.514592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:33.526162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1128/2285] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:40:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:40:33.596073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc188837-ae24-4fc6-97ec-4984ef53c560 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:33.611258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:33.614516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:33.614803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:33.615157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] PATH_INFO: `/v3/users/b6f164e7a0e146bcbada0391a542f1fe` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:33.617554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6f164e7a0e146bcbada0391a542f1fe)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:33.620946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:34.011844 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6f164e7a0e146bcbada0391a542f1fe'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:34.016757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6f164e7a0e146bcbada0391a542f1fe'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:34.044214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6f164e7a0e146bcbada0391a542f1fe'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:34.125574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6f164e7a0e146bcbada0391a542f1fe'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:34.126722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dda812af-dd34-4ad5-8be5-3a5b5832bd94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6f164e7a0e146bcbada0391a542f1fe'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:34.232732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1158/2286] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:33 2022] DELETE /identity/v3/users/b6f164e7a0e146bcbada0391a542f1fe => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:40:34.769757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:35.591035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:35.594159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:35.649835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e9685-0ef6-421f-abc6-bf0702b7ec55 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:35.888025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:35.891693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:35.892354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:35.893163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] PATH_INFO: `/v3/projects/113dae4e9c7f47b29f7da269b1a6c642` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:35.897085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=113dae4e9c7f47b29f7da269b1a6c642)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:35.915905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:36.327890 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '113dae4e9c7f47b29f7da269b1a6c642', 'request_id': 'req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:36.359167 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '113dae4e9c7f47b29f7da269b1a6c642', 'request_id': 'req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:36.394222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '113dae4e9c7f47b29f7da269b1a6c642', 'request_id': 'req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:36.524700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] The token cache is being invalidate because project 113dae4e9c7f47b29f7da269b1a6c642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:36.525194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:36.527187 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-5d622ff0-e2fa-43be-9eb8-c24f1d370e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:36.531825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1129/2287] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:35 2022] DELETE /identity/v3/projects/113dae4e9c7f47b29f7da269b1a6c642 => generated 0 bytes in 943 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:36.539005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c7fc0-2687-44fd-9294-02b775459db6 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:36.539983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c7fc0-2687-44fd-9294-02b775459db6 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:36.540709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c7fc0-2687-44fd-9294-02b775459db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:36.623887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a02c7fc0-2687-44fd-9294-02b775459db6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:36.631116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a02c7fc0-2687-44fd-9294-02b775459db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:36.892004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1159/2288] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:36.898013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bd1c1-35b8-4307-9803-5588e091f0ef None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:36.898533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bd1c1-35b8-4307-9803-5588e091f0ef None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:36.898799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bd1c1-35b8-4307-9803-5588e091f0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:36.941436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-db8bd1c1-35b8-4307-9803-5588e091f0ef None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:36.942834 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-db8bd1c1-35b8-4307-9803-5588e091f0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:36.965310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1130/2289] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:36.971236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c7fc0-2687-44fd-9294-02b775459db6 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:36.987351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c7fc0-2687-44fd-9294-02b775459db6 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:37.001756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:37.002970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:37.005186 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:37.005895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:37.006791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:40:37.008130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:37.012364 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:37.075056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1160/2290] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:40:37.079989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bd1c1-35b8-4307-9803-5588e091f0ef None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:37.099362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bd1c1-35b8-4307-9803-5588e091f0ef None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:37.109600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20107a75-b155-414e-9d3b-434e6b45042d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:37.110926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20107a75-b155-414e-9d3b-434e6b45042d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:37.111654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20107a75-b155-414e-9d3b-434e6b45042d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:37.112239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20107a75-b155-414e-9d3b-434e6b45042d admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:37.113194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20107a75-b155-414e-9d3b-434e6b45042d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:37.116766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20107a75-b155-414e-9d3b-434e6b45042d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:37.243794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:37.250737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1131/2291] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:37 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:37.288458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531d45e-a6d2-4d79-93a5-250f4e461f37 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:37.444006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6150-f83a-4083-ac9d-0b70303c2f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:37.444728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6150-f83a-4083-ac9d-0b70303c2f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:37.445015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6150-f83a-4083-ac9d-0b70303c2f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:37.445292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6150-f83a-4083-ac9d-0b70303c2f82 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:37.446352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad6150-f83a-4083-ac9d-0b70303c2f82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:37.448815 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad6150-f83a-4083-ac9d-0b70303c2f82 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:37.564122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1161/2292] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:37 2022] POST /identity/v3/users => generated 287 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:37.570683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20107a75-b155-414e-9d3b-434e6b45042d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:37.592517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20107a75-b155-414e-9d3b-434e6b45042d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:37.603710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097fb43-81cf-442d-bdcd-c4f6450a0ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:37.604732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097fb43-81cf-442d-bdcd-c4f6450a0ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:37.605520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097fb43-81cf-442d-bdcd-c4f6450a0ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:37.606028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097fb43-81cf-442d-bdcd-c4f6450a0ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:37.607830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097fb43-81cf-442d-bdcd-c4f6450a0ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:37.610330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097fb43-81cf-442d-bdcd-c4f6450a0ffe admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:37.630298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6150-f83a-4083-ac9d-0b70303c2f82 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:37.641266 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1132/2293] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:37.656138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad6150-f83a-4083-ac9d-0b70303c2f82 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:37.677925 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:37.695392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd56e05-713e-47f8-93fb-5ed07ad9e95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:37.697541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd56e05-713e-47f8-93fb-5ed07ad9e95f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:37.698157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd56e05-713e-47f8-93fb-5ed07ad9e95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:37.699940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd56e05-713e-47f8-93fb-5ed07ad9e95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:37.701468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd56e05-713e-47f8-93fb-5ed07ad9e95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:37.713066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd56e05-713e-47f8-93fb-5ed07ad9e95f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:37.778733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1162/2294] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:37.783695 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097fb43-81cf-442d-bdcd-c4f6450a0ffe admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:37.800517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097fb43-81cf-442d-bdcd-c4f6450a0ffe admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:37.818825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:37.819786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:37.820166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:37.820627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] PATH_INFO: `/v3/projects/1804d01585764cf7bb02bd332a66573e/users/16d0510b86c6444480ef4447a8a0a997/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:37.874983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1804d01585764cf7bb02bd332a66573e, user_id=16d0510b86c6444480ef4447a8a0a997, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:37.878289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:38.011989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1133/2295] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:37 2022] PUT /identity/v3/projects/1804d01585764cf7bb02bd332a66573e/users/16d0510b86c6444480ef4447a8a0a997/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:38.026866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a400cd-c1b7-4426-8291-71f83ca44910 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:38.027225 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a400cd-c1b7-4426-8291-71f83ca44910 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:38.027432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a400cd-c1b7-4426-8291-71f83ca44910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:38.084062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-31a400cd-c1b7-4426-8291-71f83ca44910 None None] MFA Rules not processed for user `16d0510b86c6444480ef4447a8a0a997`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:38.085555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-31a400cd-c1b7-4426-8291-71f83ca44910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:38.293989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1163/2296] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:38.307676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:38.311438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:38.433653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd81e1-c4a6-4263-9036-b6937e2e8a70 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:38.440534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505805b1-8200-4fd5-b213-5deb6a87cb08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:38.441261 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505805b1-8200-4fd5-b213-5deb6a87cb08 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:38.447063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505805b1-8200-4fd5-b213-5deb6a87cb08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:38.447063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505805b1-8200-4fd5-b213-5deb6a87cb08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:38.460886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505805b1-8200-4fd5-b213-5deb6a87cb08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:38.464457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505805b1-8200-4fd5-b213-5deb6a87cb08 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:38.481010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1134/2297] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:38.827619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:40.018796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:41.304218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:42.370011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca0dc9-8247-4d5e-91d2-9e30c99309a9 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:42.370011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca0dc9-8247-4d5e-91d2-9e30c99309a9 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:42.371068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca0dc9-8247-4d5e-91d2-9e30c99309a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:42.439406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0cca0dc9-8247-4d5e-91d2-9e30c99309a9 None None] MFA Rules not processed for user `16d0510b86c6444480ef4447a8a0a997`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:42.445429 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0cca0dc9-8247-4d5e-91d2-9e30c99309a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:42.495775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1164/2298] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:40:42.631585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:44.414201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:44.610187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:45.787721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7854-b1a2-4e70-b11f-b0cf5f145375 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:45.788499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7854-b1a2-4e70-b11f-b0cf5f145375 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:45.788499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7854-b1a2-4e70-b11f-b0cf5f145375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:45.855738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:45.882497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e26c7854-b1a2-4e70-b11f-b0cf5f145375 None None] MFA Rules not processed for user `2730741d853a4cfcab1bbc990cf42611`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:45.883303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e26c7854-b1a2-4e70-b11f-b0cf5f145375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:46.088067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1135/2299] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:46.095217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca0dc9-8247-4d5e-91d2-9e30c99309a9 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:46.097279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0cca0dc9-8247-4d5e-91d2-9e30c99309a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:46.302044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca0dc9-8247-4d5e-91d2-9e30c99309a9 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:46.318920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1a33-a90a-451d-8e1a-8714e1e9cbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:46.319911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1a33-a90a-451d-8e1a-8714e1e9cbbe admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:46.322328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1a33-a90a-451d-8e1a-8714e1e9cbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:46.323578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e1a33-a90a-451d-8e1a-8714e1e9cbbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:46.329480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e1a33-a90a-451d-8e1a-8714e1e9cbbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:46.334779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e1a33-a90a-451d-8e1a-8714e1e9cbbe admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:46.527746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1165/2300] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:46 2022] POST /identity/v3/projects => generated 373 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:46.534172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c7854-b1a2-4e70-b11f-b0cf5f145375 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:46.557356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c7854-b1a2-4e70-b11f-b0cf5f145375 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:46.766772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ec68e-a0e8-4de5-a2e9-0f11af46cd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:46.768092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ec68e-a0e8-4de5-a2e9-0f11af46cd6d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:46.768490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ec68e-a0e8-4de5-a2e9-0f11af46cd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:46.768780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ec68e-a0e8-4de5-a2e9-0f11af46cd6d admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:46.770029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ec68e-a0e8-4de5-a2e9-0f11af46cd6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:46.772733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ec68e-a0e8-4de5-a2e9-0f11af46cd6d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:46.931613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:46.994528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1136/2301] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:46 2022] POST /identity/v3/users => generated 293 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:40:46.998982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1a33-a90a-451d-8e1a-8714e1e9cbbe admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:47.018969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e1a33-a90a-451d-8e1a-8714e1e9cbbe admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:47.051088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4902-1f44-40c6-ba53-8df4d444ccf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:47.052196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4902-1f44-40c6-ba53-8df4d444ccf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:47.052662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4902-1f44-40c6-ba53-8df4d444ccf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:47.055142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4902-1f44-40c6-ba53-8df4d444ccf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:47.056049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff4902-1f44-40c6-ba53-8df4d444ccf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:47.058281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff4902-1f44-40c6-ba53-8df4d444ccf1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:47.095135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1166/2302] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:47.098678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ec68e-a0e8-4de5-a2e9-0f11af46cd6d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:47.135788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ec68e-a0e8-4de5-a2e9-0f11af46cd6d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:47.144552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79715653-6e1c-4dbe-9b0b-67305b272235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:47.145279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79715653-6e1c-4dbe-9b0b-67305b272235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:47.145564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79715653-6e1c-4dbe-9b0b-67305b272235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:47.146482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79715653-6e1c-4dbe-9b0b-67305b272235 admin admin] PATH_INFO: `/v3/projects/375bbf2a674b43e2927d3b4ab3f6f886/users/cc286bfe31e04fc89d411e1bdcc54a84/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:47.175993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79715653-6e1c-4dbe-9b0b-67305b272235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375bbf2a674b43e2927d3b4ab3f6f886, user_id=cc286bfe31e04fc89d411e1bdcc54a84, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:47.179593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79715653-6e1c-4dbe-9b0b-67305b272235 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:47.303561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1137/2303] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:47 2022] PUT /identity/v3/projects/375bbf2a674b43e2927d3b4ab3f6f886/users/cc286bfe31e04fc89d411e1bdcc54a84/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:47.307704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4902-1f44-40c6-ba53-8df4d444ccf1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:47.326591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4902-1f44-40c6-ba53-8df4d444ccf1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:47.511229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3217e36-be39-439e-b35e-66da632331a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:47.512026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3217e36-be39-439e-b35e-66da632331a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:47.512475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3217e36-be39-439e-b35e-66da632331a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:47.513847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3217e36-be39-439e-b35e-66da632331a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:47.515243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3217e36-be39-439e-b35e-66da632331a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:47.533541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3217e36-be39-439e-b35e-66da632331a2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:47.586224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1167/2304] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:47.588532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79715653-6e1c-4dbe-9b0b-67305b272235 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:47.606344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79715653-6e1c-4dbe-9b0b-67305b272235 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:47.626092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c52e59-ce9b-4a97-8f87-9702d5b5256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:47.628557 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52e59-ce9b-4a97-8f87-9702d5b5256b admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:47.630871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52e59-ce9b-4a97-8f87-9702d5b5256b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:47.633311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52e59-ce9b-4a97-8f87-9702d5b5256b admin admin] PATH_INFO: `/v3/projects/375bbf2a674b43e2927d3b4ab3f6f886/users/cc286bfe31e04fc89d411e1bdcc54a84/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:47.637064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52e59-ce9b-4a97-8f87-9702d5b5256b admin admin] RBAC: Authorizing `identity:create_grant(project_id=375bbf2a674b43e2927d3b4ab3f6f886, user_id=cc286bfe31e04fc89d411e1bdcc54a84, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:47.642393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52e59-ce9b-4a97-8f87-9702d5b5256b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:47.674045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1138/2305] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:47 2022] PUT /identity/v3/projects/375bbf2a674b43e2927d3b4ab3f6f886/users/cc286bfe31e04fc89d411e1bdcc54a84/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:47.932156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:48.363631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f631f3b-409c-4e03-a4d2-18b19b76e2b0 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:48.364397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f631f3b-409c-4e03-a4d2-18b19b76e2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:48.364959 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f631f3b-409c-4e03-a4d2-18b19b76e2b0 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:48.368195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1168/2306] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:40:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:40:48.378848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d298f-b0df-4c56-8076-996952aa89ba None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:48.379455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d298f-b0df-4c56-8076-996952aa89ba None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:48.380233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d298f-b0df-4c56-8076-996952aa89ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:48.445880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b64d298f-b0df-4c56-8076-996952aa89ba None None] MFA Rules not processed for user `ea29f79ed2af4341ab6f5ba50a47d8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:48.447793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b64d298f-b0df-4c56-8076-996952aa89ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:48.567379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1139/2307] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:40:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:48.580293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f631f3b-409c-4e03-a4d2-18b19b76e2b0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:48.582281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5f631f3b-409c-4e03-a4d2-18b19b76e2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:48.603961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f631f3b-409c-4e03-a4d2-18b19b76e2b0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:48.617553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe16c6b-040c-4f41-82f5-9a9982c51072 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:48.618458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe16c6b-040c-4f41-82f5-9a9982c51072 service placement] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:48.619358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe16c6b-040c-4f41-82f5-9a9982c51072 service placement] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:48.619835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe16c6b-040c-4f41-82f5-9a9982c51072 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:48.631858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe16c6b-040c-4f41-82f5-9a9982c51072 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:48.634329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe16c6b-040c-4f41-82f5-9a9982c51072 service placement] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:48.652686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1169/2308] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:40:49.836004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:51.042599 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:51.618662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:51.627595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d298f-b0df-4c56-8076-996952aa89ba None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:51.629961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b64d298f-b0df-4c56-8076-996952aa89ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:51.818713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d298f-b0df-4c56-8076-996952aa89ba None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:51.837238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:51.837942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:51.838228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:51.838508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] PATH_INFO: `/v3/users/554f6d5f3e7f420fb2212bc5185f5b20` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:51.840055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] RBAC: Authorizing `identity:delete_user(user_id=554f6d5f3e7f420fb2212bc5185f5b20)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:51.842245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:52.175696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '554f6d5f3e7f420fb2212bc5185f5b20'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:52.176957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '554f6d5f3e7f420fb2212bc5185f5b20'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:52.192811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '554f6d5f3e7f420fb2212bc5185f5b20'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:52.193317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '554f6d5f3e7f420fb2212bc5185f5b20'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:52.352092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '554f6d5f3e7f420fb2212bc5185f5b20'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:52.422565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1140/2309] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:51 2022] DELETE /identity/v3/users/554f6d5f3e7f420fb2212bc5185f5b20 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:53.317908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe16c6b-040c-4f41-82f5-9a9982c51072 service placement] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:53.421891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe16c6b-040c-4f41-82f5-9a9982c51072 service placement] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:53.744068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:53.744068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:53.744068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:53.744068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] PATH_INFO: `/v3/projects/e9fac55b82fc47fc94aae510737d4369` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:53.745087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9fac55b82fc47fc94aae510737d4369)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:53.758976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:53.995879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9fac55b82fc47fc94aae510737d4369', 'request_id': 'req-070c9c40-c0a4-4a01-a137-64dd38333283'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.040671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9fac55b82fc47fc94aae510737d4369', 'request_id': 'req-070c9c40-c0a4-4a01-a137-64dd38333283'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.222518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9fac55b82fc47fc94aae510737d4369', 'request_id': 'req-070c9c40-c0a4-4a01-a137-64dd38333283'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.245586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] The token cache is being invalidate because project e9fac55b82fc47fc94aae510737d4369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:54.246313 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.249242 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.252066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1170/2310] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:53 2022] DELETE /identity/v3/projects/e9fac55b82fc47fc94aae510737d4369 => generated 0 bytes in 937 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:54.256611 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:54.259047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:54.298372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836eff93-e3c5-4f1d-ac07-8b308dbbd12e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:54.402783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:54.405077 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:54.405946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:54.406445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] PATH_INFO: `/v3/users/dc91904a779b4288985d17f5bf79fbb0` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:54.408608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc91904a779b4288985d17f5bf79fbb0)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:54.411397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:54.767395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc91904a779b4288985d17f5bf79fbb0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.768144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc91904a779b4288985d17f5bf79fbb0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.790406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc91904a779b4288985d17f5bf79fbb0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.790406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc91904a779b4288985d17f5bf79fbb0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.798007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:54.860621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e0fb3be9-4510-4df7-8a4b-ac132c97490c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc91904a779b4288985d17f5bf79fbb0'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:54.900891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1141/2311] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:54 2022] DELETE /identity/v3/users/dc91904a779b4288985d17f5bf79fbb0 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:55.362928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:56.521134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:56.523328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:56.550539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c9c40-c0a4-4a01-a137-64dd38333283 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:56.819373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:56.820500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:56.821042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:56.822319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] PATH_INFO: `/v3/projects/3688a3a1a4274476a01dca2c895e2146` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:56.824856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3688a3a1a4274476a01dca2c895e2146)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:56.839044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:57.089377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3688a3a1a4274476a01dca2c895e2146', 'request_id': 'req-dafcad52-4c30-40a6-9f10-d13b6d9ae946'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:57.106163 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3688a3a1a4274476a01dca2c895e2146', 'request_id': 'req-dafcad52-4c30-40a6-9f10-d13b6d9ae946'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:57.265055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3688a3a1a4274476a01dca2c895e2146', 'request_id': 'req-dafcad52-4c30-40a6-9f10-d13b6d9ae946'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:57.312735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] The token cache is being invalidate because project 3688a3a1a4274476a01dca2c895e2146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:40:57.313013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:57.313728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dafcad52-4c30-40a6-9f10-d13b6d9ae946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:57.315566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1171/2312] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:40:56 2022] DELETE /identity/v3/projects/3688a3a1a4274476a01dca2c895e2146 => generated 0 bytes in 797 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:57.322129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf8ea4-6fe2-4391-ba3e-151bb86e738f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:57.323337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf8ea4-6fe2-4391-ba3e-151bb86e738f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:57.323803 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf8ea4-6fe2-4391-ba3e-151bb86e738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:57.406118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-eddf8ea4-6fe2-4391-ba3e-151bb86e738f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:57.409651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-eddf8ea4-6fe2-4391-ba3e-151bb86e738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:57.501418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1142/2313] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:57.509869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde4acc-c293-42a0-b2bc-24776a4892ce None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:57.510847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde4acc-c293-42a0-b2bc-24776a4892ce None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:57.511866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde4acc-c293-42a0-b2bc-24776a4892ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:57.540051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ebde4acc-c293-42a0-b2bc-24776a4892ce None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:57.541339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ebde4acc-c293-42a0-b2bc-24776a4892ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:57.588992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1172/2314] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:57.590371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf8ea4-6fe2-4391-ba3e-151bb86e738f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:57.613154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf8ea4-6fe2-4391-ba3e-151bb86e738f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:57.619111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:57.619805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:57.620121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:57.620674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:57.621420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:40:57.621736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:57.624408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:57.646469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1143/2315] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:40:57.648107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde4acc-c293-42a0-b2bc-24776a4892ce None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:57.664574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde4acc-c293-42a0-b2bc-24776a4892ce None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:57.684978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42209-8785-41d2-a162-edda0a60993d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:57.685615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42209-8785-41d2-a162-edda0a60993d admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:57.685898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42209-8785-41d2-a162-edda0a60993d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:57.686171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c42209-8785-41d2-a162-edda0a60993d admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:57.686823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c42209-8785-41d2-a162-edda0a60993d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:57.694337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c42209-8785-41d2-a162-edda0a60993d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:57.879367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1173/2316] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:40:57 2022] POST /identity/v3/projects => generated 361 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:57.887127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:57.946729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff9dfe-6065-4e76-87a5-d72875b24a12 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:58.191907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b49de0-ee5b-4534-9da2-bdcbc205d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:58.192904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49de0-ee5b-4534-9da2-bdcbc205d6ac admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:58.193477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49de0-ee5b-4534-9da2-bdcbc205d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:58.193934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49de0-ee5b-4534-9da2-bdcbc205d6ac admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:58.194742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b49de0-ee5b-4534-9da2-bdcbc205d6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:58.202242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b49de0-ee5b-4534-9da2-bdcbc205d6ac admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:58.421744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1144/2317] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:40:57 2022] POST /identity/v3/users => generated 281 bytes in 537 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:40:58.426275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42209-8785-41d2-a162-edda0a60993d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:58.456207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c42209-8785-41d2-a162-edda0a60993d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:58.492162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d731c68-7877-4f0d-bbdd-92ffb926f2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:58.493066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731c68-7877-4f0d-bbdd-92ffb926f2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:58.493461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731c68-7877-4f0d-bbdd-92ffb926f2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:58.493860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731c68-7877-4f0d-bbdd-92ffb926f2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:58.494634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d731c68-7877-4f0d-bbdd-92ffb926f2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:58.498021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d731c68-7877-4f0d-bbdd-92ffb926f2cd admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:58.529359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1174/2318] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:58.533228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b49de0-ee5b-4534-9da2-bdcbc205d6ac admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:58.540697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:58.550291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b49de0-ee5b-4534-9da2-bdcbc205d6ac admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:58.562703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d606da-8bcb-4ba4-a94e-8b1f84b4692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:58.563757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d606da-8bcb-4ba4-a94e-8b1f84b4692c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:58.565278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d606da-8bcb-4ba4-a94e-8b1f84b4692c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:58.565810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d606da-8bcb-4ba4-a94e-8b1f84b4692c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:58.566655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d606da-8bcb-4ba4-a94e-8b1f84b4692c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:58.571301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d606da-8bcb-4ba4-a94e-8b1f84b4692c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:58.594092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1145/2319] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:40:58 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:40:58.598774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d731c68-7877-4f0d-bbdd-92ffb926f2cd admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:58.616562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d731c68-7877-4f0d-bbdd-92ffb926f2cd admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:58.628659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:58.629805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:58.630879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:58.631390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] PATH_INFO: `/v3/projects/ecfdf6771bf043e7abfd9b99b92257d0/users/f3c90b0be5e5479eae320979916ca7c2/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:58.658260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecfdf6771bf043e7abfd9b99b92257d0, user_id=f3c90b0be5e5479eae320979916ca7c2, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:58.662425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:58.821408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1175/2320] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:40:58 2022] PUT /identity/v3/projects/ecfdf6771bf043e7abfd9b99b92257d0/users/f3c90b0be5e5479eae320979916ca7c2/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:40:58.836254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2190-6538-42ae-99f3-3408f23d2465 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:58.836881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2190-6538-42ae-99f3-3408f23d2465 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:58.837275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba2190-6538-42ae-99f3-3408f23d2465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:58.924218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-12ba2190-6538-42ae-99f3-3408f23d2465 None None] MFA Rules not processed for user `f3c90b0be5e5479eae320979916ca7c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:40:58.926068 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-12ba2190-6538-42ae-99f3-3408f23d2465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:59.158215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:40:59.180320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1146/2321] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:40:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:40:59.231309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:59.235294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:40:59.391276 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0bae7-7b26-48ca-86fa-03da9efcb5f9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:59.406403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e36bb3-3c03-4b49-a446-1077303311cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:59.407027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36bb3-3c03-4b49-a446-1077303311cc service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:59.407519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36bb3-3c03-4b49-a446-1077303311cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:59.408465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e36bb3-3c03-4b49-a446-1077303311cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:59.438223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e36bb3-3c03-4b49-a446-1077303311cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:59.440725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e36bb3-3c03-4b49-a446-1077303311cc service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:59.478840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1176/2322] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:40:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:40:59.494805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba2190-6538-42ae-99f3-3408f23d2465 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:40:59.626294 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba2190-6538-42ae-99f3-3408f23d2465 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:40:59.640516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:40:59.641184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:40:59.643317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:40:59.643317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] PATH_INFO: `/v3/users/f3c90b0be5e5479eae320979916ca7c2` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:40:59.643819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3c90b0be5e5479eae320979916ca7c2)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:40:59.645973 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:40:59.878144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3c90b0be5e5479eae320979916ca7c2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:59.878986 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3c90b0be5e5479eae320979916ca7c2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:59.898766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3c90b0be5e5479eae320979916ca7c2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:40:59.899835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3c90b0be5e5479eae320979916ca7c2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:00.047658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3c90b0be5e5479eae320979916ca7c2'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:00.081974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1147/2323] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:40:59 2022] DELETE /identity/v3/users/f3c90b0be5e5479eae320979916ca7c2 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:41:00.084031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4ab4-5444-436c-b36c-ed5f4ecb0dbc None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:00.089123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4ab4-5444-436c-b36c-ed5f4ecb0dbc None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:00.089937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4ab4-5444-436c-b36c-ed5f4ecb0dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:00.118356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9ddd4ab4-5444-436c-b36c-ed5f4ecb0dbc None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:00.119750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9ddd4ab4-5444-436c-b36c-ed5f4ecb0dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:00.245210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1177/2324] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:00.263256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:00.265557 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:00.400635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b80e6-1a1e-44d3-9387-b24bc786c849 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:00.413690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bec74a-17af-42ae-ae0a-b67b263b6466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:00.415161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec74a-17af-42ae-ae0a-b67b263b6466 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:00.415161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec74a-17af-42ae-ae0a-b67b263b6466 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:00.415161 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bec74a-17af-42ae-ae0a-b67b263b6466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:00.437040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec74a-17af-42ae-ae0a-b67b263b6466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:00.439425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bec74a-17af-42ae-ae0a-b67b263b6466 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:00.468131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1148/2325] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:41:01.062598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:01.428992 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd4ab4-5444-436c-b36c-ed5f4ecb0dbc None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:01.432366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9ddd4ab4-5444-436c-b36c-ed5f4ecb0dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:01.454498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd4ab4-5444-436c-b36c-ed5f4ecb0dbc None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:01.461760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:01.462547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:01.462898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:01.463252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] PATH_INFO: `/v3/projects/ecfdf6771bf043e7abfd9b99b92257d0` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:01.465469 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecfdf6771bf043e7abfd9b99b92257d0)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:01.470378 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:01.526548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:01.688056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecfdf6771bf043e7abfd9b99b92257d0', 'request_id': 'req-88912290-41d4-41f2-a96f-dc80136e98b2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:01.697698 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecfdf6771bf043e7abfd9b99b92257d0', 'request_id': 'req-88912290-41d4-41f2-a96f-dc80136e98b2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:01.786130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecfdf6771bf043e7abfd9b99b92257d0', 'request_id': 'req-88912290-41d4-41f2-a96f-dc80136e98b2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:01.805332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] The token cache is being invalidate because project ecfdf6771bf043e7abfd9b99b92257d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:41:01.805759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:01.806562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-88912290-41d4-41f2-a96f-dc80136e98b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:01.808301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1178/2326] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:41:01 2022] DELETE /identity/v3/projects/ecfdf6771bf043e7abfd9b99b92257d0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:41:01.815270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2ab71-a2c1-4eef-8dde-65476d5789fc None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:01.815559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2ab71-a2c1-4eef-8dde-65476d5789fc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:01.815840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2ab71-a2c1-4eef-8dde-65476d5789fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:01.843532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-32b2ab71-a2c1-4eef-8dde-65476d5789fc None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:01.845475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-32b2ab71-a2c1-4eef-8dde-65476d5789fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:01.974793 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1149/2327] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:41:01.979124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d854-e76f-46b6-8ff4-e462975393e8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:01.979645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d854-e76f-46b6-8ff4-e462975393e8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:01.980153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d854-e76f-46b6-8ff4-e462975393e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.026446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2fb4d854-e76f-46b6-8ff4-e462975393e8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:02.029571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2fb4d854-e76f-46b6-8ff4-e462975393e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:02.044147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1179/2328] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:02.050232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2ab71-a2c1-4eef-8dde-65476d5789fc None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:02.069194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2ab71-a2c1-4eef-8dde-65476d5789fc None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:02.080745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:02.081814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.082313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.082932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.084274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:41:02.097485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:02.099358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:02.127827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1150/2329] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:41:02.134021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d854-e76f-46b6-8ff4-e462975393e8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:02.160682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d854-e76f-46b6-8ff4-e462975393e8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:02.173775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d8f16-4af9-4e32-a7e8-de409573387b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:02.174783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d8f16-4af9-4e32-a7e8-de409573387b admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.175238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d8f16-4af9-4e32-a7e8-de409573387b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.175669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d8f16-4af9-4e32-a7e8-de409573387b admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.177004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d8f16-4af9-4e32-a7e8-de409573387b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:02.182030 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d8f16-4af9-4e32-a7e8-de409573387b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:02.385201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1180/2330] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:41:02 2022] POST /identity/v3/projects => generated 371 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:41:02.390952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:02.403871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85a8e9-2dc8-4900-8b3c-c6d67670f53e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:02.500923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811783-8041-4874-b0a8-bf41f0730597 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.501501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811783-8041-4874-b0a8-bf41f0730597 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.502132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa811783-8041-4874-b0a8-bf41f0730597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.502919 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c71a3-757f-4301-94fd-a76bb029c9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:02.503970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c71a3-757f-4301-94fd-a76bb029c9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.504510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c71a3-757f-4301-94fd-a76bb029c9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.505020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c71a3-757f-4301-94fd-a76bb029c9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.506281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c71a3-757f-4301-94fd-a76bb029c9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:02.509238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c71a3-757f-4301-94fd-a76bb029c9b3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:02.554582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fa811783-8041-4874-b0a8-bf41f0730597 None None] MFA Rules not processed for user `cc286bfe31e04fc89d411e1bdcc54a84`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:02.568201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fa811783-8041-4874-b0a8-bf41f0730597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:02.627763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1151/2331] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:41:02 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:41:02.635155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c71a3-757f-4301-94fd-a76bb029c9b3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:02.656411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c71a3-757f-4301-94fd-a76bb029c9b3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:02.665211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb28083-ef96-45b8-94fe-2e14fd98a410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:02.666168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb28083-ef96-45b8-94fe-2e14fd98a410 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.666642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb28083-ef96-45b8-94fe-2e14fd98a410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.667087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb28083-ef96-45b8-94fe-2e14fd98a410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.668152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb28083-ef96-45b8-94fe-2e14fd98a410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:02.671696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb28083-ef96-45b8-94fe-2e14fd98a410 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:02.687534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1152/2332] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:41:02.692597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb28083-ef96-45b8-94fe-2e14fd98a410 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:02.717473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1181/2333] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:41:02.722796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb28083-ef96-45b8-94fe-2e14fd98a410 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:02.730886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b79b-2659-4ccc-9d92-d8b4d7917cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:02.734358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa811783-8041-4874-b0a8-bf41f0730597 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:02.734653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b79b-2659-4ccc-9d92-d8b4d7917cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.734772 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fa811783-8041-4874-b0a8-bf41f0730597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:02.734901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b79b-2659-4ccc-9d92-d8b4d7917cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.734901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b79b-2659-4ccc-9d92-d8b4d7917cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.736305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769b79b-2659-4ccc-9d92-d8b4d7917cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:02.740140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769b79b-2659-4ccc-9d92-d8b4d7917cf6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:02.767712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1153/2334] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:41:02.770155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b79b-2659-4ccc-9d92-d8b4d7917cf6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:02.786268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b79b-2659-4ccc-9d92-d8b4d7917cf6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:02.809063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf9946-dea3-4cdf-85fe-886ecc5b0ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:02.810740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf9946-dea3-4cdf-85fe-886ecc5b0ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.811225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf9946-dea3-4cdf-85fe-886ecc5b0ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.811527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf9946-dea3-4cdf-85fe-886ecc5b0ad2 admin admin] PATH_INFO: `/v3/projects/d4d7964000cf4d59b147f2c5c3a1ebd9/users/a00658a834ce4262bb1de2275fe6199d/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.845518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf9946-dea3-4cdf-85fe-886ecc5b0ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d7964000cf4d59b147f2c5c3a1ebd9, user_id=a00658a834ce4262bb1de2275fe6199d, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:02.849377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa811783-8041-4874-b0a8-bf41f0730597 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:02.852133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf9946-dea3-4cdf-85fe-886ecc5b0ad2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:02.856777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:02.858012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.858012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.858395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.860673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:02.932165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1154/2335] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:41:02 2022] PUT /identity/v3/projects/d4d7964000cf4d59b147f2c5c3a1ebd9/users/a00658a834ce4262bb1de2275fe6199d/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:41:02.949394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98765479-92a4-407d-8123-a109123a55a3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:02.949672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98765479-92a4-407d-8123-a109123a55a3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:02.949825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98765479-92a4-407d-8123-a109123a55a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:02.972278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:02.974197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:02.986096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1182/2336] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:41:03.041427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-98765479-92a4-407d-8123-a109123a55a3 None None] MFA Rules not processed for user `a00658a834ce4262bb1de2275fe6199d`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:03.043734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-98765479-92a4-407d-8123-a109123a55a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:03.167125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1155/2337] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:03.189231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:03.192295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:03.379679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f638d-b958-4b83-a6fd-a4c6bab4177d service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:03.523819 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5c6fcfde34c05a2fec0659dfd54a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:03.526748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:03.527520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:03.528318 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:03.550663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:03.553854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:03.561431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a1d9f-3efd-4311-8ed1-1f9a56718ccc None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:03.561930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a1d9f-3efd-4311-8ed1-1f9a56718ccc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:03.562415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a1d9f-3efd-4311-8ed1-1f9a56718ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:03.589088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1183/2338] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:41:03.617492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:03.630813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-be5a1d9f-3efd-4311-8ed1-1f9a56718ccc None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:03.633361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-be5a1d9f-3efd-4311-8ed1-1f9a56718ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:03.714830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1156/2339] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:41:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:03.736325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:03.739462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:03.896339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:03.901975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104e269-12e1-48ca-841e-728e0d28cce0 service glance] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:03.909407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b590810-edea-484c-8bde-9629c2ed58d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:03.910404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b590810-edea-484c-8bde-9629c2ed58d4 service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:03.910857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b590810-edea-484c-8bde-9629c2ed58d4 service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:03.911487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b590810-edea-484c-8bde-9629c2ed58d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:03.929377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b590810-edea-484c-8bde-9629c2ed58d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:03.935174 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b590810-edea-484c-8bde-9629c2ed58d4 service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:03.951254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1184/2340] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:41:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:41:04.862215 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:05.456368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:07.192620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:07.447515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:08.484629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:09.552481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6173c5-9484-44fc-b027-4baecc38d9f3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:09.553402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6173c5-9484-44fc-b027-4baecc38d9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:09.557492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6173c5-9484-44fc-b027-4baecc38d9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:09.623499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-7d6173c5-9484-44fc-b027-4baecc38d9f3 None None] MFA Rules not processed for user `2730741d853a4cfcab1bbc990cf42611`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:09.624893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7d6173c5-9484-44fc-b027-4baecc38d9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:09.717685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:09.819879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1157/2341] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:41:10.378894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:10.694137 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:10.730808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:11.352524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:13.901075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695ee0-271e-417f-8ff7-71d8dcd84dff None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:13.902155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695ee0-271e-417f-8ff7-71d8dcd84dff None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:13.902777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695ee0-271e-417f-8ff7-71d8dcd84dff None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:13.905407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1185/2342] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:41:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:41:13.918522 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca119cc-5f52-48a7-97d5-df8ecd408de3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:13.919417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca119cc-5f52-48a7-97d5-df8ecd408de3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:13.919871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca119cc-5f52-48a7-97d5-df8ecd408de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:13.944546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2ca119cc-5f52-48a7-97d5-df8ecd408de3 None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:13.946447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2ca119cc-5f52-48a7-97d5-df8ecd408de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:14.040891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1158/2343] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:41:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:20.861649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=58 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:22.650216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c695ee0-271e-417f-8ff7-71d8dcd84dff None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:22.662735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c695ee0-271e-417f-8ff7-71d8dcd84dff None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:22.666454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74abfa-8455-4462-afed-b50a07fe159d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5c6fcfde34c05a2fec0659dfd54a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:22.667286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74abfa-8455-4462-afed-b50a07fe159d service glance] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:22.667643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74abfa-8455-4462-afed-b50a07fe159d service glance] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:22.667921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74abfa-8455-4462-afed-b50a07fe159d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:22.695796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74abfa-8455-4462-afed-b50a07fe159d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:22.697417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74abfa-8455-4462-afed-b50a07fe159d service glance] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:22.710488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1186/2344] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:41:23.239624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399305-876b-411c-aa93-67e96ad686e9 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:23.240779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399305-876b-411c-aa93-67e96ad686e9 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:23.241285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c399305-876b-411c-aa93-67e96ad686e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:23.280569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6c399305-876b-411c-aa93-67e96ad686e9 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:23.284093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80f79-ccbf-4ee8-8036-c2679a4283d0 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:23.285187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80f79-ccbf-4ee8-8036-c2679a4283d0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:23.286132 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa80f79-ccbf-4ee8-8036-c2679a4283d0 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:23.289368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1187/2345] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:41:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:41:23.292488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6c399305-876b-411c-aa93-67e96ad686e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:23.306256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86509-4a50-424d-9f80-b1787bcaa34a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:23.307696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86509-4a50-424d-9f80-b1787bcaa34a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:23.310519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c86509-4a50-424d-9f80-b1787bcaa34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:23.325461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1159/2346] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:41:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:23.351280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c399305-876b-411c-aa93-67e96ad686e9 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:23.363309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f8c86509-4a50-424d-9f80-b1787bcaa34a None None] MFA Rules not processed for user `18ca375f53414475b75dae308cd19fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:23.365337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f8c86509-4a50-424d-9f80-b1787bcaa34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:23.374560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c399305-876b-411c-aa93-67e96ad686e9 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:23.382082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:23.383848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:23.384374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:23.384804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:23.396331 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:23.398362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:23.409775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1160/2347] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:41:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:41:23.473644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1188/2348] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:41:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:41:23.493543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:23.496231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:23.518256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:23.531516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be5185-d254-4bac-9eb7-c1bb6e7aeb4c service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:23.538251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55ea58-0fba-47a4-88e4-541937af28ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:23.539252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55ea58-0fba-47a4-88e4-541937af28ea service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:23.539775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55ea58-0fba-47a4-88e4-541937af28ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:23.540306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55ea58-0fba-47a4-88e4-541937af28ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:23.558797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55ea58-0fba-47a4-88e4-541937af28ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:23.562563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55ea58-0fba-47a4-88e4-541937af28ea service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:23.579708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1161/2349] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:41:23.685601 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a541c5-3a8c-4db1-9a72-407260ae89fd None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:23.686396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a541c5-3a8c-4db1-9a72-407260ae89fd None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:23.686396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a541c5-3a8c-4db1-9a72-407260ae89fd None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:23.688401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1189/2350] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:41:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:41:23.696263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6a1a-1085-4ce4-8d7c-a9358b3087ec None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:23.696570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6a1a-1085-4ce4-8d7c-a9358b3087ec None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:23.696817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda6a1a-1085-4ce4-8d7c-a9358b3087ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:23.723232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9bda6a1a-1085-4ce4-8d7c-a9358b3087ec None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:23.724518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9bda6a1a-1085-4ce4-8d7c-a9358b3087ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:23.743204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1162/2351] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:41:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:23.751926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a541c5-3a8c-4db1-9a72-407260ae89fd None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:23.756337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a6a541c5-3a8c-4db1-9a72-407260ae89fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:23.776645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a541c5-3a8c-4db1-9a72-407260ae89fd None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:23.781069 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff914b4-560a-4a7d-a05f-ea96bb1f8371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:23.781689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff914b4-560a-4a7d-a05f-ea96bb1f8371 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:23.782133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff914b4-560a-4a7d-a05f-ea96bb1f8371 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:23.782562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff914b4-560a-4a7d-a05f-ea96bb1f8371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:23.794275 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff914b4-560a-4a7d-a05f-ea96bb1f8371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:23.796074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff914b4-560a-4a7d-a05f-ea96bb1f8371 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:23.809858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1190/2352] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:41:25.101314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedab11-0fc8-41ef-8028-e1f19a6e6d1c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:25.102287 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedab11-0fc8-41ef-8028-e1f19a6e6d1c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:25.103160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedab11-0fc8-41ef-8028-e1f19a6e6d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:25.132114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8cedab11-0fc8-41ef-8028-e1f19a6e6d1c None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:25.133657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8cedab11-0fc8-41ef-8028-e1f19a6e6d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:25.142487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1163/2353] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 13:41:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:25.157640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff914b4-560a-4a7d-a05f-ea96bb1f8371 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:25.170966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff914b4-560a-4a7d-a05f-ea96bb1f8371 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:25.176987 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dea29-c5d9-4848-bc3d-86f893424749 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:25.177896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dea29-c5d9-4848-bc3d-86f893424749 service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:25.178453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dea29-c5d9-4848-bc3d-86f893424749 service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:25.178893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dea29-c5d9-4848-bc3d-86f893424749 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:25.204209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dea29-c5d9-4848-bc3d-86f893424749 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:25.208080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dea29-c5d9-4848-bc3d-86f893424749 service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:25.243382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1191/2354] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:41:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:41:26.233277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d1ac2-e9f4-4b9c-a4d1-b598ad5da7d6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:26.233831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d1ac2-e9f4-4b9c-a4d1-b598ad5da7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:26.233947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d1ac2-e9f4-4b9c-a4d1-b598ad5da7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:26.257967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2f1d1ac2-e9f4-4b9c-a4d1-b598ad5da7d6 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:26.259409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2f1d1ac2-e9f4-4b9c-a4d1-b598ad5da7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:26.268457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1164/2355] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:41:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:41:26.284607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dea29-c5d9-4848-bc3d-86f893424749 service swift] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:26.298522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dea29-c5d9-4848-bc3d-86f893424749 service swift] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:26.304986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435cc75-3b32-4cb6-8557-1da1527ab5ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:26.305942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435cc75-3b32-4cb6-8557-1da1527ab5ca service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:26.306460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435cc75-3b32-4cb6-8557-1da1527ab5ca service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:26.307160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435cc75-3b32-4cb6-8557-1da1527ab5ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:26.317408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435cc75-3b32-4cb6-8557-1da1527ab5ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:26.320083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435cc75-3b32-4cb6-8557-1da1527ab5ca service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:26.331565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1192/2356] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 13:41:28.687882 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d1ac2-e9f4-4b9c-a4d1-b598ad5da7d6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:28.689900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2f1d1ac2-e9f4-4b9c-a4d1-b598ad5da7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:28.770690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d1ac2-e9f4-4b9c-a4d1-b598ad5da7d6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:28.775947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:28.776597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:28.778024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:28.778024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:28.779804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:28.792046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:28.794781 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:28.804914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1165/2357] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:41:32.059758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51993c66-12b9-4f9e-8cd4-64cb04265c63 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:32.060974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51993c66-12b9-4f9e-8cd4-64cb04265c63 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:32.060974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51993c66-12b9-4f9e-8cd4-64cb04265c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:32.082832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-51993c66-12b9-4f9e-8cd4-64cb04265c63 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:32.084084 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-51993c66-12b9-4f9e-8cd4-64cb04265c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:32.092342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1193/2358] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 13:41:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:32.109596 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:32.123940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692687-da33-4d68-b1ce-7706702569b5 service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:32.130806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5dfa6-e625-4586-9985-391eba8b8192 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:32.134958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5dfa6-e625-4586-9985-391eba8b8192 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:32.134958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5dfa6-e625-4586-9985-391eba8b8192 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:32.134958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5dfa6-e625-4586-9985-391eba8b8192 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:32.145227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5dfa6-e625-4586-9985-391eba8b8192 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:32.147627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5dfa6-e625-4586-9985-391eba8b8192 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:32.157701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1166/2359] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:41:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:41:33.115365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe1da-3fb7-4338-bb0e-f1de4e36037a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:33.115843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe1da-3fb7-4338-bb0e-f1de4e36037a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:33.115936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe1da-3fb7-4338-bb0e-f1de4e36037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:33.142316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b4abe1da-3fb7-4338-bb0e-f1de4e36037a None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:33.144532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b4abe1da-3fb7-4338-bb0e-f1de4e36037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:33.151916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1194/2360] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:41:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:33.177907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5dfa6-e625-4586-9985-391eba8b8192 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:33.204595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5dfa6-e625-4586-9985-391eba8b8192 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:33.213181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17094e-61c6-4027-8fbf-661349bee926 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:33.213932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17094e-61c6-4027-8fbf-661349bee926 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:33.214269 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17094e-61c6-4027-8fbf-661349bee926 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:33.214710 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17094e-61c6-4027-8fbf-661349bee926 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:33.227513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17094e-61c6-4027-8fbf-661349bee926 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:33.232213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17094e-61c6-4027-8fbf-661349bee926 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:33.248857 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1167/2361] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:41:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:41:36.731942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abe1da-3fb7-4338-bb0e-f1de4e36037a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:36.733595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b4abe1da-3fb7-4338-bb0e-f1de4e36037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:36.869873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abe1da-3fb7-4338-bb0e-f1de4e36037a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:36.875542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:36.876348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:36.876733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:36.877112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] PATH_INFO: `/v3/users/16d0510b86c6444480ef4447a8a0a997` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:36.878410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d0510b86c6444480ef4447a8a0a997)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:36.881022 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:37.324297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:37.326167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d0510b86c6444480ef4447a8a0a997'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:37.326882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d0510b86c6444480ef4447a8a0a997'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:37.335012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d0510b86c6444480ef4447a8a0a997'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:37.608153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d0510b86c6444480ef4447a8a0a997'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:37.609172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d0510b86c6444480ef4447a8a0a997'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:37.642321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1195/2362] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:41:36 2022] DELETE /identity/v3/users/16d0510b86c6444480ef4447a8a0a997 => generated 0 bytes in 907 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:41:37.643347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dc5ef-8326-439a-8385-ec424f73a07b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:37.643863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dc5ef-8326-439a-8385-ec424f73a07b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:37.644019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dc5ef-8326-439a-8385-ec424f73a07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:37.667897 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4c8dc5ef-8326-439a-8385-ec424f73a07b None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:37.669206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4c8dc5ef-8326-439a-8385-ec424f73a07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:37.802899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1168/2363] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:37.825720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:37.827258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:37.943627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df6403-5680-4b3f-9643-c18a296ccc16 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:37.959885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b29ef9-5907-4bc9-bc72-cdfa38b3e219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:37.960537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b29ef9-5907-4bc9-bc72-cdfa38b3e219 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:37.961017 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b29ef9-5907-4bc9-bc72-cdfa38b3e219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:37.961295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b29ef9-5907-4bc9-bc72-cdfa38b3e219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:37.987591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b29ef9-5907-4bc9-bc72-cdfa38b3e219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:37.994420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b29ef9-5907-4bc9-bc72-cdfa38b3e219 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:38.004224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1196/2364] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:41:38.378665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dc5ef-8326-439a-8385-ec424f73a07b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:38.381840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4c8dc5ef-8326-439a-8385-ec424f73a07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:38.419812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dc5ef-8326-439a-8385-ec424f73a07b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:38.429011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:38.430105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:38.430537 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:38.431002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] PATH_INFO: `/v3/projects/1804d01585764cf7bb02bd332a66573e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:38.433359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1804d01585764cf7bb02bd332a66573e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:38.438214 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:38.567944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:38.587076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1804d01585764cf7bb02bd332a66573e', 'request_id': 'req-ae402499-e5ba-4b59-9bd8-3b8b149ab891'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:38.598497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1804d01585764cf7bb02bd332a66573e', 'request_id': 'req-ae402499-e5ba-4b59-9bd8-3b8b149ab891'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:38.611265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1804d01585764cf7bb02bd332a66573e', 'request_id': 'req-ae402499-e5ba-4b59-9bd8-3b8b149ab891'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:38.723148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] The token cache is being invalidate because project 1804d01585764cf7bb02bd332a66573e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:41:38.724332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:38.724432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:38.726234 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1169/2365] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:41:38 2022] DELETE /identity/v3/projects/1804d01585764cf7bb02bd332a66573e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:41:41.870441 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68ff9c-efd9-4394-a7a3-89fb57e872c0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:41.872078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68ff9c-efd9-4394-a7a3-89fb57e872c0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:41.872715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68ff9c-efd9-4394-a7a3-89fb57e872c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:41.906702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bc68ff9c-efd9-4394-a7a3-89fb57e872c0 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:41.908667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc68ff9c-efd9-4394-a7a3-89fb57e872c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:42.035676 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1197/2366] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:41:42.056898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:42.056898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:42.123494 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae402499-e5ba-4b59-9bd8-3b8b149ab891 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:42.129943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3ad78-d332-4c15-9e67-c3e9a98c7d4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:42.130690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ad78-d332-4c15-9e67-c3e9a98c7d4d service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:42.131116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ad78-d332-4c15-9e67-c3e9a98c7d4d service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:42.131575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ad78-d332-4c15-9e67-c3e9a98c7d4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:42.142874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3ad78-d332-4c15-9e67-c3e9a98c7d4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:42.144556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3ad78-d332-4c15-9e67-c3e9a98c7d4d service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:42.155132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1170/2367] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:41:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:41:45.774535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68ff9c-efd9-4394-a7a3-89fb57e872c0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:45.776145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc68ff9c-efd9-4394-a7a3-89fb57e872c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:45.853548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68ff9c-efd9-4394-a7a3-89fb57e872c0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:45.858513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:45.859150 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:45.859427 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:45.859694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] PATH_INFO: `/v3/users/a00658a834ce4262bb1de2275fe6199d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:45.860865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a00658a834ce4262bb1de2275fe6199d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:45.862829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:45.878204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:46.076039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a00658a834ce4262bb1de2275fe6199d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:46.077227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a00658a834ce4262bb1de2275fe6199d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:46.088481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a00658a834ce4262bb1de2275fe6199d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:46.150906 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a00658a834ce4262bb1de2275fe6199d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:46.151768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a00658a834ce4262bb1de2275fe6199d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:46.173760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1198/2368] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:41:45 2022] DELETE /identity/v3/users/a00658a834ce4262bb1de2275fe6199d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:41:46.178659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20938-7755-42d2-985a-a431d2e44f24 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:46.179391 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20938-7755-42d2-985a-a431d2e44f24 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:46.179826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc20938-7755-42d2-985a-a431d2e44f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:46.218779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-abc20938-7755-42d2-985a-a431d2e44f24 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:46.220782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-abc20938-7755-42d2-985a-a431d2e44f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:46.303700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1171/2369] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:46.317178 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:46.318946 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:46.403528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a2c39-6801-49ea-a264-b4a34fa82bb9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:46.409379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:46.410011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:46.410398 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:46.410677 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:46.439519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:46.442397 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:46.455290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1199/2370] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:41:46.955665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc20938-7755-42d2-985a-a431d2e44f24 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:46.958797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-abc20938-7755-42d2-985a-a431d2e44f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:46.976133 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc20938-7755-42d2-985a-a431d2e44f24 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:46.990444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:46.993491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:47.000346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:47.001190 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:47.001448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:47.001870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] PATH_INFO: `/v3/users/e9c7ecfdb9904095b122bb8fd82121fe` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:47.003450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9c7ecfdb9904095b122bb8fd82121fe)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:47.005515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:47.061343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa976eba-460d-4061-b2ec-910f628fc81f service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:47.071096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:47.072228 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:47.072738 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:47.073236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] PATH_INFO: `/v3/projects/d4d7964000cf4d59b147f2c5c3a1ebd9` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:47.077272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d7964000cf4d59b147f2c5c3a1ebd9)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:47.081189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:47.328310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d7964000cf4d59b147f2c5c3a1ebd9', 'request_id': 'req-1e5f9f0f-bc83-47d6-8487-d932590a36d9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.343596 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:47.354056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9c7ecfdb9904095b122bb8fd82121fe'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.359761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9c7ecfdb9904095b122bb8fd82121fe'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.363850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d7964000cf4d59b147f2c5c3a1ebd9', 'request_id': 'req-1e5f9f0f-bc83-47d6-8487-d932590a36d9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.369915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9c7ecfdb9904095b122bb8fd82121fe'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.372425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9c7ecfdb9904095b122bb8fd82121fe'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.472161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d7964000cf4d59b147f2c5c3a1ebd9', 'request_id': 'req-1e5f9f0f-bc83-47d6-8487-d932590a36d9'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.498103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] The token cache is being invalidate because project d4d7964000cf4d59b147f2c5c3a1ebd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:41:47.498435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.499669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.501943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1200/2371] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:41:46 2022] DELETE /identity/v3/projects/d4d7964000cf4d59b147f2c5c3a1ebd9 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:41:47.563165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbbede3e-b863-4c75-b0f9-47617d9ca297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9c7ecfdb9904095b122bb8fd82121fe'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:47.594378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1172/2372] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:41:46 2022] DELETE /identity/v3/users/e9c7ecfdb9904095b122bb8fd82121fe => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:41:48.009528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:48.013096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:48.083241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:48.179301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f9f0f-bc83-47d6-8487-d932590a36d9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:48.264610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:48.265541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:48.265990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:48.266440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] PATH_INFO: `/v3/projects/c22fb394e99345d88b2e082b22902af3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:48.274129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22fb394e99345d88b2e082b22902af3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:48.276241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:48.472039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22fb394e99345d88b2e082b22902af3', 'request_id': 'req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:48.482246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22fb394e99345d88b2e082b22902af3', 'request_id': 'req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:48.618235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22fb394e99345d88b2e082b22902af3', 'request_id': 'req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:48.632368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] The token cache is being invalidate because project c22fb394e99345d88b2e082b22902af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:41:48.632685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:48.633447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-828a96e3-1e74-44f4-bcb5-4b2c6d6818b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:41:48.635426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1201/2373] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:41:48 2022] DELETE /identity/v3/projects/c22fb394e99345d88b2e082b22902af3 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:41:48.642487 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c2f1e-e57b-43aa-9ad7-ce068fc138b7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:48.643393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c2f1e-e57b-43aa-9ad7-ce068fc138b7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:48.643842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c2f1e-e57b-43aa-9ad7-ce068fc138b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:48.689422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b98c2f1e-e57b-43aa-9ad7-ce068fc138b7 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:48.690676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b98c2f1e-e57b-43aa-9ad7-ce068fc138b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:48.777928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1173/2374] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:41:48.786747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d30e10-73b7-46ed-8aa3-87f51113c559 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:48.787035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d30e10-73b7-46ed-8aa3-87f51113c559 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:48.787311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d30e10-73b7-46ed-8aa3-87f51113c559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:48.813889 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-33d30e10-73b7-46ed-8aa3-87f51113c559 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:48.815151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-33d30e10-73b7-46ed-8aa3-87f51113c559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:48.839893 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1202/2375] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:41:48.843163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c2f1e-e57b-43aa-9ad7-ce068fc138b7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:48.861129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c2f1e-e57b-43aa-9ad7-ce068fc138b7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:48.873122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:48.873745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:48.874086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:48.874594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:48.877208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:41:48.877492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:48.879545 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:48.898932 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1174/2376] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:41:48.899570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d30e10-73b7-46ed-8aa3-87f51113c559 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:48.910746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d30e10-73b7-46ed-8aa3-87f51113c559 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:48.934780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9124e6-aa43-4d2c-a26a-b1f2d2074ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:48.935976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9124e6-aa43-4d2c-a26a-b1f2d2074ffe admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:48.936299 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9124e6-aa43-4d2c-a26a-b1f2d2074ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:48.936583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9124e6-aa43-4d2c-a26a-b1f2d2074ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:48.937241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9124e6-aa43-4d2c-a26a-b1f2d2074ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:48.939565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9124e6-aa43-4d2c-a26a-b1f2d2074ffe admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:49.108931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1203/2377] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:41:48 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:41:49.111176 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:49.124706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af82ecb-4ab4-404c-adf3-4cfedc21c3f8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:49.193130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcafed1-0751-4a2a-9b89-9ed2ad2e15de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:49.193740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcafed1-0751-4a2a-9b89-9ed2ad2e15de admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:49.194017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcafed1-0751-4a2a-9b89-9ed2ad2e15de admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:49.194310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcafed1-0751-4a2a-9b89-9ed2ad2e15de admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:49.194933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcafed1-0751-4a2a-9b89-9ed2ad2e15de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:49.196891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcafed1-0751-4a2a-9b89-9ed2ad2e15de admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:49.342539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1175/2378] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:41:49 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:41:49.346623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9124e6-aa43-4d2c-a26a-b1f2d2074ffe admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:49.359544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9124e6-aa43-4d2c-a26a-b1f2d2074ffe admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:49.364877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7dc99-f110-45a0-9424-b9c2f5a2e691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:49.365580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7dc99-f110-45a0-9424-b9c2f5a2e691 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:49.365934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7dc99-f110-45a0-9424-b9c2f5a2e691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:49.366302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7dc99-f110-45a0-9424-b9c2f5a2e691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:49.367041 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7dc99-f110-45a0-9424-b9c2f5a2e691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:49.368817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7dc99-f110-45a0-9424-b9c2f5a2e691 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:49.388566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1204/2379] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:41:49.392994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcafed1-0751-4a2a-9b89-9ed2ad2e15de admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:49.406027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcafed1-0751-4a2a-9b89-9ed2ad2e15de admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:49.411183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f0ab0-2a6c-437c-88b8-906fbdf7f38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:49.411902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f0ab0-2a6c-437c-88b8-906fbdf7f38d admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:49.412293 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f0ab0-2a6c-437c-88b8-906fbdf7f38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:49.412649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f0ab0-2a6c-437c-88b8-906fbdf7f38d admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:49.413447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f0ab0-2a6c-437c-88b8-906fbdf7f38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:49.415325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f0ab0-2a6c-437c-88b8-906fbdf7f38d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:49.436900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1176/2380] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:41:49.441525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7dc99-f110-45a0-9424-b9c2f5a2e691 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:49.458226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7dc99-f110-45a0-9424-b9c2f5a2e691 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:49.468233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:49.469158 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:49.469627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:49.470093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] PATH_INFO: `/v3/projects/1a87479c006646c8a96f1cc9d6f81af8/users/b98d3f39e1c846c9895e22bbb029a447/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:49.487478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:49.494298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a87479c006646c8a96f1cc9d6f81af8, user_id=b98d3f39e1c846c9895e22bbb029a447, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:49.510685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:49.586526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1205/2381] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:41:49 2022] PUT /identity/v3/projects/1a87479c006646c8a96f1cc9d6f81af8/users/b98d3f39e1c846c9895e22bbb029a447/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:41:49.599396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9693e-3334-484e-ac33-c05d8bda59df None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:49.602448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9693e-3334-484e-ac33-c05d8bda59df None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:49.602448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9693e-3334-484e-ac33-c05d8bda59df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:49.650730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c6c9693e-3334-484e-ac33-c05d8bda59df None None] MFA Rules not processed for user `b98d3f39e1c846c9895e22bbb029a447`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:41:49.653739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c6c9693e-3334-484e-ac33-c05d8bda59df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:49.675836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:49.753122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1177/2382] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:41:49.766305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:41:49.768306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:41:49.878442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f51e0-48c4-4da7-a58c-68d6db2ebb74 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:41:49.883910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:41:49.884587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:41:49.884873 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:41:49.885156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:41:49.897721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:41:49.899756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:41:49.910668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1206/2383] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:41:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:41:59.888884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=20 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:41:59.906428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:13.281003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7813e-b793-453c-af2b-8b5df1262559 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:13.281576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7813e-b793-453c-af2b-8b5df1262559 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:13.281576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7813e-b793-453c-af2b-8b5df1262559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:13.304274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4bf7813e-b793-453c-af2b-8b5df1262559 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:13.305618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4bf7813e-b793-453c-af2b-8b5df1262559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:13.440703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1178/2384] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 13:42:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:42:13.449973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:13.453055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:13.574048 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece8e0-a663-469b-b516-263b89b34bbd service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:13.579425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed02e0-d03c-4799-bdba-1a736d6ccf75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:13.580070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed02e0-d03c-4799-bdba-1a736d6ccf75 service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:13.580622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed02e0-d03c-4799-bdba-1a736d6ccf75 service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:13.580778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed02e0-d03c-4799-bdba-1a736d6ccf75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:13.592335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed02e0-d03c-4799-bdba-1a736d6ccf75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:13.594194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed02e0-d03c-4799-bdba-1a736d6ccf75 service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:13.608553 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1207/2385] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:42:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 13:42:15.039497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7813e-b793-453c-af2b-8b5df1262559 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:15.110681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7813e-b793-453c-af2b-8b5df1262559 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:15.115350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:15.115983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:15.116297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:15.116577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] PATH_INFO: `/v3/users/b98d3f39e1c846c9895e22bbb029a447` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:15.117723 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b98d3f39e1c846c9895e22bbb029a447)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:15.119729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:15.312739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b98d3f39e1c846c9895e22bbb029a447'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:15.313443 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b98d3f39e1c846c9895e22bbb029a447'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:15.321365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b98d3f39e1c846c9895e22bbb029a447'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:15.322228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b98d3f39e1c846c9895e22bbb029a447'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:15.476660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b98d3f39e1c846c9895e22bbb029a447'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:15.500040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1179/2386] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:42:15 2022] DELETE /identity/v3/users/b98d3f39e1c846c9895e22bbb029a447 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:42:15.505670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca902856-6773-4935-82d8-be0629270c76 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:15.506515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca902856-6773-4935-82d8-be0629270c76 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:15.506515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca902856-6773-4935-82d8-be0629270c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:15.535538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ca902856-6773-4935-82d8-be0629270c76 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:15.536804 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ca902856-6773-4935-82d8-be0629270c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:15.648993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1208/2387] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:42:15.660232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:15.662335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:15.704681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:15.758827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8698c6-80f3-47c9-9bcd-1a69a7e16764 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:15.765444 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72844144-7a22-4d32-8e0f-e45cb7a5779c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:15.766074 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72844144-7a22-4d32-8e0f-e45cb7a5779c service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:15.766389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72844144-7a22-4d32-8e0f-e45cb7a5779c service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:15.767748 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72844144-7a22-4d32-8e0f-e45cb7a5779c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:15.784203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72844144-7a22-4d32-8e0f-e45cb7a5779c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:15.785743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72844144-7a22-4d32-8e0f-e45cb7a5779c service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:15.806190 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1180/2388] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:42:16.862009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca902856-6773-4935-82d8-be0629270c76 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:16.862009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ca902856-6773-4935-82d8-be0629270c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:16.875663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca902856-6773-4935-82d8-be0629270c76 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:16.882697 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:16.884586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:16.885673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:16.886142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] PATH_INFO: `/v3/projects/1a87479c006646c8a96f1cc9d6f81af8` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:16.887609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a87479c006646c8a96f1cc9d6f81af8)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:16.889858 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:17.107535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a87479c006646c8a96f1cc9d6f81af8', 'request_id': 'req-f4f7e706-47ec-489e-aa9e-5af921d2575d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:17.116561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a87479c006646c8a96f1cc9d6f81af8', 'request_id': 'req-f4f7e706-47ec-489e-aa9e-5af921d2575d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:17.254625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a87479c006646c8a96f1cc9d6f81af8', 'request_id': 'req-f4f7e706-47ec-489e-aa9e-5af921d2575d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:17.277042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] The token cache is being invalidate because project 1a87479c006646c8a96f1cc9d6f81af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:42:17.277820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:17.279529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4f7e706-47ec-489e-aa9e-5af921d2575d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:42:17.282576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1209/2389] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:42:16 2022] DELETE /identity/v3/projects/1a87479c006646c8a96f1cc9d6f81af8 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:42:17.305548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5382b9-af0f-4d0d-9ea4-bf48bf77e5d6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:17.306824 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5382b9-af0f-4d0d-9ea4-bf48bf77e5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:17.308070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5382b9-af0f-4d0d-9ea4-bf48bf77e5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:17.335607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-cd5382b9-af0f-4d0d-9ea4-bf48bf77e5d6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:17.337090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cd5382b9-af0f-4d0d-9ea4-bf48bf77e5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:17.428806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1181/2390] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:42:17.433026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489db57c-041d-486a-bffd-4bb114ab7b3c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:17.433683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489db57c-041d-486a-bffd-4bb114ab7b3c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:17.434206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489db57c-041d-486a-bffd-4bb114ab7b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:17.463799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-489db57c-041d-486a-bffd-4bb114ab7b3c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:17.465538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-489db57c-041d-486a-bffd-4bb114ab7b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:17.479793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1210/2391] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:42:17.481445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5382b9-af0f-4d0d-9ea4-bf48bf77e5d6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:17.499928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5382b9-af0f-4d0d-9ea4-bf48bf77e5d6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:17.509786 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:17.510771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:17.511157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:17.511640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:17.513111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:42:17.513614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:17.516871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:17.546014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1182/2392] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:42:17.546852 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489db57c-041d-486a-bffd-4bb114ab7b3c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:17.560137 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489db57c-041d-486a-bffd-4bb114ab7b3c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:17.568410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b056b-c09c-4bf8-abbe-a1fbbe7a80b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:17.569281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b056b-c09c-4bf8-abbe-a1fbbe7a80b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:17.569645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b056b-c09c-4bf8-abbe-a1fbbe7a80b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:17.569980 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b056b-c09c-4bf8-abbe-a1fbbe7a80b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:17.570883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b056b-c09c-4bf8-abbe-a1fbbe7a80b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:17.573468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b056b-c09c-4bf8-abbe-a1fbbe7a80b2 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:17.580485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:17.704395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1211/2393] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:42:17 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:42:17.705785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:17.728902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe1199-18c4-4de7-a117-4c2fa36b91ac admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:17.831258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c499e-7ba7-4486-8d44-948712a22d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:17.831929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c499e-7ba7-4486-8d44-948712a22d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:17.832228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c499e-7ba7-4486-8d44-948712a22d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:17.832649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c499e-7ba7-4486-8d44-948712a22d46 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:17.833326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c499e-7ba7-4486-8d44-948712a22d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:17.836135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c499e-7ba7-4486-8d44-948712a22d46 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:17.953644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1183/2394] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:42:17 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:42:17.958108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b056b-c09c-4bf8-abbe-a1fbbe7a80b2 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:17.970888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b056b-c09c-4bf8-abbe-a1fbbe7a80b2 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:17.977208 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f7c9d-12b9-4390-a5dd-3b6fa79fa2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:17.978040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f7c9d-12b9-4390-a5dd-3b6fa79fa2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:17.978414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f7c9d-12b9-4390-a5dd-3b6fa79fa2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:17.979124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f7c9d-12b9-4390-a5dd-3b6fa79fa2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:17.979981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f7c9d-12b9-4390-a5dd-3b6fa79fa2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:17.981881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f7c9d-12b9-4390-a5dd-3b6fa79fa2d7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:17.994128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1212/2395] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:42:17.999049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c499e-7ba7-4486-8d44-948712a22d46 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:18.011346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c499e-7ba7-4486-8d44-948712a22d46 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:18.017318 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9f145-a1eb-4158-b990-2a89e93b22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:18.017966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f145-a1eb-4158-b990-2a89e93b22ac admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:18.018246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f145-a1eb-4158-b990-2a89e93b22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:18.018546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f145-a1eb-4158-b990-2a89e93b22ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:18.019207 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9f145-a1eb-4158-b990-2a89e93b22ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:18.020874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9f145-a1eb-4158-b990-2a89e93b22ac admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:18.045779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1184/2396] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:42:18.050647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f7c9d-12b9-4390-a5dd-3b6fa79fa2d7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:18.062311 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f7c9d-12b9-4390-a5dd-3b6fa79fa2d7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:18.067554 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:18.070293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:18.070293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:18.070293 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] PATH_INFO: `/v3/projects/6842f55d3dcc42289b6bfe02ed020dc4/users/98b30f0436864263b2829921f212f8ce/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:18.093726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6842f55d3dcc42289b6bfe02ed020dc4, user_id=98b30f0436864263b2829921f212f8ce, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:18.098792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:18.269576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1213/2397] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:42:18 2022] PUT /identity/v3/projects/6842f55d3dcc42289b6bfe02ed020dc4/users/98b30f0436864263b2829921f212f8ce/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:42:18.275009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2de355-9b82-41ce-904a-b2b9d21fc78c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:18.275304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2de355-9b82-41ce-904a-b2b9d21fc78c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:18.275585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2de355-9b82-41ce-904a-b2b9d21fc78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:18.303739 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2b2de355-9b82-41ce-904a-b2b9d21fc78c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:18.305365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2b2de355-9b82-41ce-904a-b2b9d21fc78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:18.360005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:18.397243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1185/2398] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:42:18.408877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:18.411546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:18.489671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32782f94-e92a-421f-a111-9366038de178 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:18.493817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51533a-593f-4a7d-81c4-6e518f140123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:18.495085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51533a-593f-4a7d-81c4-6e518f140123 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:18.495548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51533a-593f-4a7d-81c4-6e518f140123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:18.496001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51533a-593f-4a7d-81c4-6e518f140123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:18.506795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51533a-593f-4a7d-81c4-6e518f140123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:18.508279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51533a-593f-4a7d-81c4-6e518f140123 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:18.518648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1214/2399] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:42:22.903098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ae21-413b-46dd-96f9-761834dd9b5b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:22.903698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ae21-413b-46dd-96f9-761834dd9b5b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:22.903989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040ae21-413b-46dd-96f9-761834dd9b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:22.946284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8040ae21-413b-46dd-96f9-761834dd9b5b None None] MFA Rules not processed for user `ea9d4eafe59c4c8180eece8ec9283e40`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:22.949595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8040ae21-413b-46dd-96f9-761834dd9b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:23.060753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1186/2400] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:42:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:42:25.134661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dafaa-0acc-4dfe-848c-db65fec48cd4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:25.135885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dafaa-0acc-4dfe-848c-db65fec48cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:25.136037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dafaa-0acc-4dfe-848c-db65fec48cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:25.161912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5c1dafaa-0acc-4dfe-848c-db65fec48cd4 None None] MFA Rules not processed for user `adb792e7246d42dface4aa52899a0ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:25.161912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5c1dafaa-0acc-4dfe-848c-db65fec48cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:25.290045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1215/2401] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:42:27.776677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723ae9-e22d-42f8-abaa-c1131d2cb66f None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:27.777575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723ae9-e22d-42f8-abaa-c1131d2cb66f None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:27.778025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5723ae9-e22d-42f8-abaa-c1131d2cb66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:27.821962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b5723ae9-e22d-42f8-abaa-c1131d2cb66f None None] MFA Rules not processed for user `98b30f0436864263b2829921f212f8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:27.824109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b5723ae9-e22d-42f8-abaa-c1131d2cb66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:27.936464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1187/2402] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:42:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:42:27.944002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dafaa-0acc-4dfe-848c-db65fec48cd4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:27.968190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dafaa-0acc-4dfe-848c-db65fec48cd4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:27.977723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a0991-6b2f-41d8-bf56-fd30c9135aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:27.979652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a0991-6b2f-41d8-bf56-fd30c9135aaf admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:27.980284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a0991-6b2f-41d8-bf56-fd30c9135aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:27.980882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a0991-6b2f-41d8-bf56-fd30c9135aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:27.982200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a0991-6b2f-41d8-bf56-fd30c9135aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:27.992443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a0991-6b2f-41d8-bf56-fd30c9135aaf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:28.165749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1216/2403] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:42:27 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:42:28.178498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5723ae9-e22d-42f8-abaa-c1131d2cb66f None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:28.208550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5723ae9-e22d-42f8-abaa-c1131d2cb66f None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:28.365888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2ab31-f05e-4596-8078-595f14dcea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:28.366534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ab31-f05e-4596-8078-595f14dcea39 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:28.366821 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ab31-f05e-4596-8078-595f14dcea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:28.367120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ab31-f05e-4596-8078-595f14dcea39 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:28.368243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ab31-f05e-4596-8078-595f14dcea39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:28.371004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ab31-f05e-4596-8078-595f14dcea39 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:28.512923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:28.524521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1188/2404] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:42:28 2022] POST /identity/v3/users => generated 295 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:42:28.527935 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a0991-6b2f-41d8-bf56-fd30c9135aaf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:28.548011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a0991-6b2f-41d8-bf56-fd30c9135aaf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:28.559328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f84ff-d47d-4091-bd33-3be733fd9da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:28.566099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f84ff-d47d-4091-bd33-3be733fd9da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:28.566099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f84ff-d47d-4091-bd33-3be733fd9da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:28.566099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f84ff-d47d-4091-bd33-3be733fd9da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:28.566099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f84ff-d47d-4091-bd33-3be733fd9da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:28.567585 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f84ff-d47d-4091-bd33-3be733fd9da7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:28.590496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1217/2405] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:42:28.594227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2ab31-f05e-4596-8078-595f14dcea39 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:28.605405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2ab31-f05e-4596-8078-595f14dcea39 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:28.610689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e4f3-ce41-4ca0-8828-21447e1555e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:28.611515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e4f3-ce41-4ca0-8828-21447e1555e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:28.629674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e4f3-ce41-4ca0-8828-21447e1555e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:28.629674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e4f3-ce41-4ca0-8828-21447e1555e7 admin admin] PATH_INFO: `/v3/projects/acc55cc2d1e34e70aacabed458a3b107/users/532f2f52948a4074abff9ea07cf09f47/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:28.642806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6e4f3-ce41-4ca0-8828-21447e1555e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc55cc2d1e34e70aacabed458a3b107, user_id=532f2f52948a4074abff9ea07cf09f47, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:28.645451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6e4f3-ce41-4ca0-8828-21447e1555e7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:28.726799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1189/2406] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:42:28 2022] PUT /identity/v3/projects/acc55cc2d1e34e70aacabed458a3b107/users/532f2f52948a4074abff9ea07cf09f47/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:42:28.729838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f84ff-d47d-4091-bd33-3be733fd9da7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:28.757122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f84ff-d47d-4091-bd33-3be733fd9da7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:28.815591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:28.897916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:28.898996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:28.899836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:28.900456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:28.903183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:28.906930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:28.932322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1218/2407] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:42:28.936903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e4f3-ce41-4ca0-8828-21447e1555e7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:28.976395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e4f3-ce41-4ca0-8828-21447e1555e7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:29.001961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86729fd9-b239-41e3-a756-71787d0afd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:29.002656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86729fd9-b239-41e3-a756-71787d0afd5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:29.002945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86729fd9-b239-41e3-a756-71787d0afd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:29.003228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86729fd9-b239-41e3-a756-71787d0afd5d admin admin] PATH_INFO: `/v3/projects/acc55cc2d1e34e70aacabed458a3b107/users/532f2f52948a4074abff9ea07cf09f47/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:29.005442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86729fd9-b239-41e3-a756-71787d0afd5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc55cc2d1e34e70aacabed458a3b107, user_id=532f2f52948a4074abff9ea07cf09f47, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:29.008230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86729fd9-b239-41e3-a756-71787d0afd5d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:29.031527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1190/2408] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:42:28 2022] PUT /identity/v3/projects/acc55cc2d1e34e70aacabed458a3b107/users/532f2f52948a4074abff9ea07cf09f47/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:42:33.441861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:33.444653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:33.546269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fcc6c-257b-4301-9f9f-d0559c91b759 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:33.619570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:33.620417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:33.620834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:33.621289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:33.623770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:33.927823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:33.927823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:33.927823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1219/2409] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:42:39.027403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=16,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:40.130646 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb1d7-c755-4161-9016-21612737b2a5 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:40.131332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb1d7-c755-4161-9016-21612737b2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:40.131332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edb1d7-c755-4161-9016-21612737b2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:40.175346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a6edb1d7-c755-4161-9016-21612737b2a5 None None] MFA Rules not processed for user `532f2f52948a4074abff9ea07cf09f47`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:42:40.178019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a6edb1d7-c755-4161-9016-21612737b2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:40.295226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1191/2410] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:42:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:42:40.324148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:42:40.325473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:42:40.371909 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:40.462244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0ab2-9875-4da3-922b-dffbc09f3559 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:42:40.471307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c23a7-267c-452a-a847-2fb647ad7053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:42:40.472661 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c23a7-267c-452a-a847-2fb647ad7053 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:42:40.473308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c23a7-267c-452a-a847-2fb647ad7053 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:42:40.473924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c23a7-267c-452a-a847-2fb647ad7053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:42:40.547871 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c23a7-267c-452a-a847-2fb647ad7053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:42:40.549796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c23a7-267c-452a-a847-2fb647ad7053 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:42:40.565032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1220/2411] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:42:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:42:50.289696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=34 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:42:50.557336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=44 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:43:04.836322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b7920-dc72-4d3f-8148-cb5560e4fe25 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:43:04.837231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b7920-dc72-4d3f-8148-cb5560e4fe25 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:43:04.837669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b7920-dc72-4d3f-8148-cb5560e4fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:43:04.859962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-682b7920-dc72-4d3f-8148-cb5560e4fe25 None None] MFA Rules not processed for user `98b30f0436864263b2829921f212f8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:43:04.861535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-682b7920-dc72-4d3f-8148-cb5560e4fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:43:04.870673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1192/2412] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:43:09.002207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ae8f7-afb0-4d90-9671-3f0ab21aecbb None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:43:09.003664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ae8f7-afb0-4d90-9671-3f0ab21aecbb None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:43:09.003664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ae8f7-afb0-4d90-9671-3f0ab21aecbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:43:09.022530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3c8ae8f7-afb0-4d90-9671-3f0ab21aecbb None None] MFA Rules not processed for user `773dc87736034257b3d1ae2e8f02bf23`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:43:09.023735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3c8ae8f7-afb0-4d90-9671-3f0ab21aecbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:43:09.083431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1221/2413] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:43:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:43:12.713302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b7920-dc72-4d3f-8148-cb5560e4fe25 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:43:12.725641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b7920-dc72-4d3f-8148-cb5560e4fe25 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:43:12.730701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:43:12.731610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:43:12.732090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:43:12.732559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:43:12.734638 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:43:12.749879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:43:12.751686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:43:12.765763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1193/2414] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:43:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:43:15.169029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ae8f7-afb0-4d90-9671-3f0ab21aecbb None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:43:15.179394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ae8f7-afb0-4d90-9671-3f0ab21aecbb None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:43:15.183863 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:43:15.184549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:43:15.184828 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:43:15.185101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:43:15.187261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:43:15.201769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:43:15.203207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:43:15.214867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1222/2415] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:43:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:43:22.760022 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=9 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:43:25.209725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=31 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:43:26.081005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:43:26.090666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de11d38-4028-407e-b61e-b9d59c4f6219 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:43:26.094449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aef15-ab3b-48bd-8584-c92e3be54c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:43:26.095044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aef15-ab3b-48bd-8584-c92e3be54c66 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:43:26.095326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aef15-ab3b-48bd-8584-c92e3be54c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:43:26.095597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aef15-ab3b-48bd-8584-c92e3be54c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:43:26.105515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aef15-ab3b-48bd-8584-c92e3be54c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:43:26.106956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aef15-ab3b-48bd-8584-c92e3be54c66 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:43:26.117488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1194/2416] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:43:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:43:33.058433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:43:33.059986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:43:33.077206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2d0c0-1882-4ba5-bfbd-1d4a789c79a7 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:43:33.083755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:43:33.086795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:43:33.086795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:43:33.086795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:43:33.087742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:43:33.149120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:43:33.150901 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:43:33.160817 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1223/2417] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:43:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:43:36.113079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:43:43.156510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=28 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:43:52.036547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aef15-ab3b-48bd-8584-c92e3be54c66 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:43:52.047452 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aef15-ab3b-48bd-8584-c92e3be54c66 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:43:52.051818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f215f-3330-411f-a91c-702177402da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:43:52.052525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f215f-3330-411f-a91c-702177402da3 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:43:52.052900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f215f-3330-411f-a91c-702177402da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:43:52.053175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f215f-3330-411f-a91c-702177402da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:43:52.056029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ae7f215f-3330-411f-a91c-702177402da3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:43:52.074688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f215f-3330-411f-a91c-702177402da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:43:52.077120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f215f-3330-411f-a91c-702177402da3 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:43:52.090120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1195/2418] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:43:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:43:56.653210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:43:56.679883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5630e-6b1c-4ad8-93ca-1685bf10b9fc service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:43:56.692907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:43:56.696243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:43:56.696243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:43:56.696243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:43:56.696842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:43:56.719906 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:43:56.724857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:43:56.739816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1224/2419] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:43:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:02.084201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:44:02.290660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b8b72-192f-4bca-9383-7d63893d3acd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:02.291018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b8b72-192f-4bca-9383-7d63893d3acd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:02.291188 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b8b72-192f-4bca-9383-7d63893d3acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:02.320780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-722b8b72-192f-4bca-9383-7d63893d3acd None None] MFA Rules not processed for user `ea9d4eafe59c4c8180eece8ec9283e40`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:44:02.322947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-722b8b72-192f-4bca-9383-7d63893d3acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:02.337223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1196/2420] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:44:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:44:02.833675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:02.845603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87c53e-58d2-4b74-b767-77cd4e43eb1f service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:02.849531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6ef7e-7170-4dbc-b59b-8e58249bb22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:02.850155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6ef7e-7170-4dbc-b59b-8e58249bb22c service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:02.850435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6ef7e-7170-4dbc-b59b-8e58249bb22c service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:02.850708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6ef7e-7170-4dbc-b59b-8e58249bb22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:02.852692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b9d6ef7e-7170-4dbc-b59b-8e58249bb22c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:02.865972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6ef7e-7170-4dbc-b59b-8e58249bb22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:02.867509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6ef7e-7170-4dbc-b59b-8e58249bb22c service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:02.878098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1225/2421] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:03.565003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b8b72-192f-4bca-9383-7d63893d3acd None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:03.567229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-722b8b72-192f-4bca-9383-7d63893d3acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:03.720290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b8b72-192f-4bca-9383-7d63893d3acd None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:03.727665 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:03.728637 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:03.729127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:03.729584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:03.732642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:03.754141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:03.756776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:03.770015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1197/2422] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:10.038000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685a3f4-9ef9-4cf7-97ba-58349c1d4b43 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:10.038000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685a3f4-9ef9-4cf7-97ba-58349c1d4b43 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:10.038000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685a3f4-9ef9-4cf7-97ba-58349c1d4b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:10.070317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c685a3f4-9ef9-4cf7-97ba-58349c1d4b43 None None] MFA Rules not processed for user `adb792e7246d42dface4aa52899a0ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:44:10.072548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c685a3f4-9ef9-4cf7-97ba-58349c1d4b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:10.154418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1226/2423] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:44:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:44:10.960797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:10.988274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea3434-f77e-4a71-b1ee-0c2cf049108b service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:10.996017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:10.997052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:10.997276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:10.997658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:11.000288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:11.016631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:11.018976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:11.034039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1198/2424] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:16.347669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685a3f4-9ef9-4cf7-97ba-58349c1d4b43 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:16.368953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685a3f4-9ef9-4cf7-97ba-58349c1d4b43 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:16.380877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a59874-5095-4743-94cb-1886630c6b36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:16.382120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a59874-5095-4743-94cb-1886630c6b36 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:16.382626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a59874-5095-4743-94cb-1886630c6b36 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:16.383120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a59874-5095-4743-94cb-1886630c6b36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:16.385947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-89a59874-5095-4743-94cb-1886630c6b36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:16.401060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a59874-5095-4743-94cb-1886630c6b36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:16.403343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a59874-5095-4743-94cb-1886630c6b36 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:16.419660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1227/2425] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:17.482423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:17.494810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc42fb4-7097-4879-92d7-1f49b585c532 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:17.502492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:17.503129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:17.503650 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:17.504136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:17.506212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:17.522286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:17.523893 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:17.539185 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1199/2426] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:26.412438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=43 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:44:27.530926 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=47 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:44:32.791540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d65f7-8986-4d0b-87c1-e3f864ff2f38 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:32.793159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d65f7-8986-4d0b-87c1-e3f864ff2f38 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:32.793159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d65f7-8986-4d0b-87c1-e3f864ff2f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:32.816418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-af2d65f7-8986-4d0b-87c1-e3f864ff2f38 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:44:32.817647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-af2d65f7-8986-4d0b-87c1-e3f864ff2f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:32.878353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1228/2427] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:44:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:44:33.044616 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:33.046774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:33.122838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6f973-aaee-4aa1-b95a-5c94cff48c15 service nova] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:33.128999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cbbb4-eef7-44c3-8e4a-2bc2924d9f6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:33.129618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cbbb4-eef7-44c3-8e4a-2bc2924d9f6a service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:33.129914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cbbb4-eef7-44c3-8e4a-2bc2924d9f6a service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:33.130191 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cbbb4-eef7-44c3-8e4a-2bc2924d9f6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:33.140093 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cbbb4-eef7-44c3-8e4a-2bc2924d9f6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:33.142040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75cbbb4-eef7-44c3-8e4a-2bc2924d9f6a service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:33.151338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1200/2428] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:44:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:44:42.870995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=25 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:44:43.147658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=32 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:44:50.516407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d65f7-8986-4d0b-87c1-e3f864ff2f38 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:50.526001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d65f7-8986-4d0b-87c1-e3f864ff2f38 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:50.530454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839487c-d5d9-4922-80a7-fd68c7f2bd1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:50.531057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839487c-d5d9-4922-80a7-fd68c7f2bd1a service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:50.531341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839487c-d5d9-4922-80a7-fd68c7f2bd1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:50.531620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839487c-d5d9-4922-80a7-fd68c7f2bd1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:50.533403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7839487c-d5d9-4922-80a7-fd68c7f2bd1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:50.543297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839487c-d5d9-4922-80a7-fd68c7f2bd1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:50.545510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839487c-d5d9-4922-80a7-fd68c7f2bd1a service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:50.553725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1229/2429] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:51.002094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa26bc-bdf2-41c7-b152-0f790c4d1206 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.002640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa26bc-bdf2-41c7-b152-0f790c4d1206 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.002640 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa26bc-bdf2-41c7-b152-0f790c4d1206 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.004146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1201/2430] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:44:51.009670 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78edd3-0bdd-4bd4-b2a5-b0ec8d69c856 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.009963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78edd3-0bdd-4bd4-b2a5-b0ec8d69c856 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.010240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78edd3-0bdd-4bd4-b2a5-b0ec8d69c856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.032707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-2e78edd3-0bdd-4bd4-b2a5-b0ec8d69c856 None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:44:51.034317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2e78edd3-0bdd-4bd4-b2a5-b0ec8d69c856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:51.044919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1230/2431] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:44:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:44:51.058428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa26bc-bdf2-41c7-b152-0f790c4d1206 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:51.060316 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-55fa26bc-bdf2-41c7-b152-0f790c4d1206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:51.136674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa26bc-bdf2-41c7-b152-0f790c4d1206 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:51.141846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c4040-0419-45b5-bebc-125d405b85b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5c6fcfde34c05a2fec0659dfd54a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:51.142560 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c4040-0419-45b5-bebc-125d405b85b9 service glance] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.142912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c4040-0419-45b5-bebc-125d405b85b9 service glance] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.143195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c4040-0419-45b5-bebc-125d405b85b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.153989 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c4040-0419-45b5-bebc-125d405b85b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:51.155992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c4040-0419-45b5-bebc-125d405b85b9 service glance] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:51.167541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1202/2432] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:51.396978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a19a9-6ba3-44cd-8f19-69be893e3269 None None] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.397273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a19a9-6ba3-44cd-8f19-69be893e3269 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.397550 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a19a9-6ba3-44cd-8f19-69be893e3269 None None] PATH_INFO: `/` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.398904 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1231/2433] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:44:51.403593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5135b-85aa-4cc1-a8ec-06bcb8332236 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.404019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5135b-85aa-4cc1-a8ec-06bcb8332236 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.404754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5135b-85aa-4cc1-a8ec-06bcb8332236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.424922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8fe5135b-85aa-4cc1-a8ec-06bcb8332236 None None] MFA Rules not processed for user `18ca375f53414475b75dae308cd19fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:44:51.426152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8fe5135b-85aa-4cc1-a8ec-06bcb8332236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:51.432829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1203/2434] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:44:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:44:51.445602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a19a9-6ba3-44cd-8f19-69be893e3269 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:51.461696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a19a9-6ba3-44cd-8f19-69be893e3269 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:51.465279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c0f0b-c0c0-43e1-ab64-139f4b7fd25e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:51.465878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c0f0b-c0c0-43e1-ab64-139f4b7fd25e service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.466159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c0f0b-c0c0-43e1-ab64-139f4b7fd25e service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.466451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c0f0b-c0c0-43e1-ab64-139f4b7fd25e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.475262 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c0f0b-c0c0-43e1-ab64-139f4b7fd25e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:51.476856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c0f0b-c0c0-43e1-ab64-139f4b7fd25e service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:51.486978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1232/2435] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:51.576288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e7ec2-910e-41bd-b2b9-b9088ad822a8 None None] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.583876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e7ec2-910e-41bd-b2b9-b9088ad822a8 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.583876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e7ec2-910e-41bd-b2b9-b9088ad822a8 None None] PATH_INFO: `/` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.583876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1204/2436] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 13:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 13:44:51.584201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7b9e-3e51-4972-8d54-39dd3b410201 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.584629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7b9e-3e51-4972-8d54-39dd3b410201 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.584737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7b9e-3e51-4972-8d54-39dd3b410201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.604495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1ffb7b9e-3e51-4972-8d54-39dd3b410201 None None] MFA Rules not processed for user `40561a306123461d92bc669be3962db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:44:51.605701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1ffb7b9e-3e51-4972-8d54-39dd3b410201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:51.615567 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1233/2437] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 13:44:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:44:51.626851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e7ec2-910e-41bd-b2b9-b9088ad822a8 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:51.638378 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e7ec2-910e-41bd-b2b9-b9088ad822a8 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:51.642097 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d21a9a-585a-4a08-8c79-eba23fc33733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:51.642772 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d21a9a-585a-4a08-8c79-eba23fc33733 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:51.643090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d21a9a-585a-4a08-8c79-eba23fc33733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:51.643370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d21a9a-585a-4a08-8c79-eba23fc33733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:51.654875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d21a9a-585a-4a08-8c79-eba23fc33733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:51.656371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d21a9a-585a-4a08-8c79-eba23fc33733 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:51.673861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1205/2438] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:44:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:44:54.467121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd375b40-85a1-44dd-a79d-41d2202177db None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:54.467681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd375b40-85a1-44dd-a79d-41d2202177db None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:54.467908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd375b40-85a1-44dd-a79d-41d2202177db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:54.493361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-bd375b40-85a1-44dd-a79d-41d2202177db None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:44:54.494936 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bd375b40-85a1-44dd-a79d-41d2202177db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:44:54.505436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1234/2439] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:44:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:44:54.518321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d21a9a-585a-4a08-8c79-eba23fc33733 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:44:54.531587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d21a9a-585a-4a08-8c79-eba23fc33733 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:44:54.538155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51f23a-8994-4469-923f-19f68cfb4a50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:44:54.538776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f23a-8994-4469-923f-19f68cfb4a50 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:44:54.539060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f23a-8994-4469-923f-19f68cfb4a50 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:44:54.539511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f23a-8994-4469-923f-19f68cfb4a50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:44:54.556291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51f23a-8994-4469-923f-19f68cfb4a50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:44:54.557928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51f23a-8994-4469-923f-19f68cfb4a50 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:44:54.567552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1206/2440] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:44:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:45:04.486948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=21 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:04.564192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=41 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:12.814950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d4d6-1758-4b68-b551-c9e3644decc0 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:12.816927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d4d6-1758-4b68-b551-c9e3644decc0 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:12.817169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d4d6-1758-4b68-b551-c9e3644decc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:12.880358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9c02d4d6-1758-4b68-b551-c9e3644decc0 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:12.881582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9c02d4d6-1758-4b68-b551-c9e3644decc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:12.890219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1235/2441] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:12.904796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51f23a-8994-4469-923f-19f68cfb4a50 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:12.924615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51f23a-8994-4469-923f-19f68cfb4a50 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:12.941578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10dbaa-57ea-476e-8686-f275daec7963 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:12.942500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10dbaa-57ea-476e-8686-f275daec7963 service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:12.943801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10dbaa-57ea-476e-8686-f275daec7963 service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:12.944145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10dbaa-57ea-476e-8686-f275daec7963 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:12.958035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10dbaa-57ea-476e-8686-f275daec7963 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:12.962356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10dbaa-57ea-476e-8686-f275daec7963 service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:12.977846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1207/2442] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:45:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:45:14.688403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02d4d6-1758-4b68-b551-c9e3644decc0 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:14.704064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02d4d6-1758-4b68-b551-c9e3644decc0 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:14.710408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:14.711441 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:14.711730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:14.712045 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:14.713985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:14.727603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:14.729350 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:14.739838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1236/2443] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:45:22.973498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:24.735039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=9 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:32.060168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10dbaa-57ea-476e-8686-f275daec7963 service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:32.070943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10dbaa-57ea-476e-8686-f275daec7963 service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:32.074586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:32.075181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:32.075459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:32.075729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:32.078664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:32.142722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:32.144286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:32.155132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1208/2444] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:45:33.965642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:33.983021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5a54e-29de-4c29-9700-e592796ce692 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:33.987755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:33.989446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:33.990040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:33.990468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:33.994039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:34.012529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:34.014279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:34.037373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1237/2445] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:45:36.820361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:36.821874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:36.842119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6360-2810-4f87-b3cc-4d8c57c1c320 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:36.847724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:36.848661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:36.849000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:36.849278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] PATH_INFO: `/v3/users/ea9d4eafe59c4c8180eece8ec9283e40` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:36.850877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9d4eafe59c4c8180eece8ec9283e40)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:36.852923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:37.123575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9d4eafe59c4c8180eece8ec9283e40'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:37.125901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9d4eafe59c4c8180eece8ec9283e40'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:37.138321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9d4eafe59c4c8180eece8ec9283e40'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:37.139295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9d4eafe59c4c8180eece8ec9283e40'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:37.211147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6f0ee578-a2f0-4590-82ac-32b91bcd5649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9d4eafe59c4c8180eece8ec9283e40'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:37.242952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1209/2446] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:45:36 2022] DELETE /identity/v3/users/ea9d4eafe59c4c8180eece8ec9283e40 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:37.981604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:38.001405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c21ec-13b3-4b17-8908-f5d537d1db7d service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:38.111917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:38.112910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:38.113370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:38.114034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] PATH_INFO: `/v3/projects/019b2e8bdb934d94a25fe4b355e73c68` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:38.115875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=019b2e8bdb934d94a25fe4b355e73c68)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:38.126996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:38.295930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019b2e8bdb934d94a25fe4b355e73c68', 'request_id': 'req-b71ddc59-982f-43de-bd48-a01242e53c5b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:38.306454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019b2e8bdb934d94a25fe4b355e73c68', 'request_id': 'req-b71ddc59-982f-43de-bd48-a01242e53c5b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:38.462374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019b2e8bdb934d94a25fe4b355e73c68', 'request_id': 'req-b71ddc59-982f-43de-bd48-a01242e53c5b'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:38.475628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] The token cache is being invalidate because project 019b2e8bdb934d94a25fe4b355e73c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:45:38.476020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:38.476806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b71ddc59-982f-43de-bd48-a01242e53c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:38.478351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1238/2447] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:45:37 2022] DELETE /identity/v3/projects/019b2e8bdb934d94a25fe4b355e73c68 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:45:38.483939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732b10-7c4a-41df-b6a9-451086e36860 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:38.484605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732b10-7c4a-41df-b6a9-451086e36860 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:38.484913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a732b10-7c4a-41df-b6a9-451086e36860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:38.509331 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-1a732b10-7c4a-41df-b6a9-451086e36860 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:38.510472 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-1a732b10-7c4a-41df-b6a9-451086e36860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:38.607449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1210/2448] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:38.617683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83734ca-881b-44d6-8130-310695c9ecf2 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:38.618818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83734ca-881b-44d6-8130-310695c9ecf2 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:38.618818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83734ca-881b-44d6-8130-310695c9ecf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:38.644247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d83734ca-881b-44d6-8130-310695c9ecf2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:38.646190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d83734ca-881b-44d6-8130-310695c9ecf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:38.658810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1239/2449] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:38.664256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a732b10-7c4a-41df-b6a9-451086e36860 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:38.675127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a732b10-7c4a-41df-b6a9-451086e36860 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:38.682361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:38.683146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:38.683748 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:38.684240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:38.684982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:45:38.689234 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:38.689234 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:38.689657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:38.702774 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1211/2450] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:45:38.707065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83734ca-881b-44d6-8130-310695c9ecf2 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:38.719042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83734ca-881b-44d6-8130-310695c9ecf2 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:38.728418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:38.729343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:38.729914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:38.730608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:38.732814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:38.734854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:38.956712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1240/2451] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:45:38 2022] POST /identity/v3/projects => generated 373 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:38.961799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:38.988901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7fdb6-d640-4a09-a3c3-6d50a4726cc4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:38.991066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:38.994119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:39.072631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:39.084130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45bce4-bca3-442a-99b0-3604802788a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:39.085038 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45bce4-bca3-442a-99b0-3604802788a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.085329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45bce4-bca3-442a-99b0-3604802788a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.085773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45bce4-bca3-442a-99b0-3604802788a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.086606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45bce4-bca3-442a-99b0-3604802788a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:39.088776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45bce4-bca3-442a-99b0-3604802788a3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:39.091857 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99598d0-bf21-461e-bf87-5c433c4aeb6e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:39.096710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:39.097592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.098814 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.099304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] PATH_INFO: `/v3/users/adb792e7246d42dface4aa52899a0ffc` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.100853 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb792e7246d42dface4aa52899a0ffc)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:39.102995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:39.239644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1212/2452] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:45:38 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:45:39.244071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45bce4-bca3-442a-99b0-3604802788a3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:39.255747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45bce4-bca3-442a-99b0-3604802788a3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:39.260943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed7c1d-2125-4a66-afac-a08d1e73761b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:39.261562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7c1d-2125-4a66-afac-a08d1e73761b admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.261833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7c1d-2125-4a66-afac-a08d1e73761b admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.262105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7c1d-2125-4a66-afac-a08d1e73761b admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.262759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed7c1d-2125-4a66-afac-a08d1e73761b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:39.264423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed7c1d-2125-4a66-afac-a08d1e73761b admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:39.281157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1213/2453] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:45:39.291587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed7c1d-2125-4a66-afac-a08d1e73761b admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:39.303418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed7c1d-2125-4a66-afac-a08d1e73761b admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:39.314582 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c94290-60a9-4efe-a2f0-5e0d612bed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:39.315358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c94290-60a9-4efe-a2f0-5e0d612bed27 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.315751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c94290-60a9-4efe-a2f0-5e0d612bed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.316122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c94290-60a9-4efe-a2f0-5e0d612bed27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.317636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c94290-60a9-4efe-a2f0-5e0d612bed27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:39.319372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c94290-60a9-4efe-a2f0-5e0d612bed27 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:39.331203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1214/2454] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:45:39.333486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8217a0-b97b-4cb6-aee7-2a0b7a48a31a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.334110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8217a0-b97b-4cb6-aee7-2a0b7a48a31a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.334298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8217a0-b97b-4cb6-aee7-2a0b7a48a31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.354711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3f8217a0-b97b-4cb6-aee7-2a0b7a48a31a None None] MFA Rules not processed for user `98b30f0436864263b2829921f212f8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:39.355991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3f8217a0-b97b-4cb6-aee7-2a0b7a48a31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:39.416251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb792e7246d42dface4aa52899a0ffc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:39.416978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb792e7246d42dface4aa52899a0ffc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:39.418745 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1215/2455] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:39.419594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8217a0-b97b-4cb6-aee7-2a0b7a48a31a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:39.427826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb792e7246d42dface4aa52899a0ffc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:39.431817 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8217a0-b97b-4cb6-aee7-2a0b7a48a31a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:39.437113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a373198-5587-406f-8999-40f7b0edf674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:39.437839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373198-5587-406f-8999-40f7b0edf674 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.438141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373198-5587-406f-8999-40f7b0edf674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.438421 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373198-5587-406f-8999-40f7b0edf674 admin admin] PATH_INFO: `/v3/projects/f7357cf5935541a7aba0fb7c46e224cc/users/eeafd7e29adf43f2a4849950931a645a/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.459302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a373198-5587-406f-8999-40f7b0edf674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7357cf5935541a7aba0fb7c46e224cc, user_id=eeafd7e29adf43f2a4849950931a645a, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:39.462183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a373198-5587-406f-8999-40f7b0edf674 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:39.576571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb792e7246d42dface4aa52899a0ffc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:39.583425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb792e7246d42dface4aa52899a0ffc'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:39.584688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1216/2456] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:39 2022] PUT /identity/v3/projects/f7357cf5935541a7aba0fb7c46e224cc/users/eeafd7e29adf43f2a4849950931a645a/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:39.595086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319a89e-63d2-40a7-8835-74a074efd621 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.595583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319a89e-63d2-40a7-8835-74a074efd621 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.595944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319a89e-63d2-40a7-8835-74a074efd621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.601927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1241/2457] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:45:38 2022] DELETE /identity/v3/users/adb792e7246d42dface4aa52899a0ffc => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:39.641876 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9319a89e-63d2-40a7-8835-74a074efd621 None None] MFA Rules not processed for user `eeafd7e29adf43f2a4849950931a645a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:39.643120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9319a89e-63d2-40a7-8835-74a074efd621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:39.678272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:39.731192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1217/2458] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:45:39.735816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:39.737361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:39.824233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb553c-eed5-46f7-9028-af90a5dab3c8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:39.829250 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578adb0-0f9e-43fb-ba46-9213e9f0ad47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:39.829881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578adb0-0f9e-43fb-ba46-9213e9f0ad47 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.830248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578adb0-0f9e-43fb-ba46-9213e9f0ad47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.830532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578adb0-0f9e-43fb-ba46-9213e9f0ad47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.831200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578adb0-0f9e-43fb-ba46-9213e9f0ad47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:39.833335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7578adb0-0f9e-43fb-ba46-9213e9f0ad47 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:39.918450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319a89e-63d2-40a7-8835-74a074efd621 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:39.919760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9319a89e-63d2-40a7-8835-74a074efd621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:39.945427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319a89e-63d2-40a7-8835-74a074efd621 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:39.950374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:39.951015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:39.951304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:39.951556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] PATH_INFO: `/v3/projects/127e5da55b164ce68568c153fee58baf` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:39.953343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=127e5da55b164ce68568c153fee58baf)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:39.955801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:39.959156 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1242/2459] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:45:39 2022] POST /identity/v3/projects => generated 373 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:39.964883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578adb0-0f9e-43fb-ba46-9213e9f0ad47 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.024502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:40.035747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578adb0-0f9e-43fb-ba46-9213e9f0ad47 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.040919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce60d7b-3b47-4e67-9a87-45016bdd2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.041542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce60d7b-3b47-4e67-9a87-45016bdd2845 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.041826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce60d7b-3b47-4e67-9a87-45016bdd2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.042103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce60d7b-3b47-4e67-9a87-45016bdd2845 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.042740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce60d7b-3b47-4e67-9a87-45016bdd2845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.044783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce60d7b-3b47-4e67-9a87-45016bdd2845 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.097037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127e5da55b164ce68568c153fee58baf', 'request_id': 'req-3bbca153-b4ac-4501-b057-b168dd0867ee'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:40.104808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127e5da55b164ce68568c153fee58baf', 'request_id': 'req-3bbca153-b4ac-4501-b057-b168dd0867ee'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:40.112359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127e5da55b164ce68568c153fee58baf', 'request_id': 'req-3bbca153-b4ac-4501-b057-b168dd0867ee'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:40.169834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1243/2460] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:45:39 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:40.173994 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce60d7b-3b47-4e67-9a87-45016bdd2845 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.186830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce60d7b-3b47-4e67-9a87-45016bdd2845 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.190324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] The token cache is being invalidate because project 127e5da55b164ce68568c153fee58baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:45:40.190813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:40.191851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3bbca153-b4ac-4501-b057-b168dd0867ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:40.194002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1218/2461] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:45:39 2022] DELETE /identity/v3/projects/127e5da55b164ce68568c153fee58baf => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:45:40.198993 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf2f72-30e8-4356-8b91-5f56126640a1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.199288 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf2f72-30e8-4356-8b91-5f56126640a1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.199591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf2f72-30e8-4356-8b91-5f56126640a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.221129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2bcf2f72-30e8-4356-8b91-5f56126640a1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:40.222380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2bcf2f72-30e8-4356-8b91-5f56126640a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:40.251595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.252232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.252518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.252795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.253449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.255282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.265524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1244/2462] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:45:40.270799 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.272641 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:40.286167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d6105-cc78-470d-b62f-1e133f1e8f06 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.291094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98205bf-a428-4de5-92cf-2f227426a398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.291767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98205bf-a428-4de5-92cf-2f227426a398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.292053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98205bf-a428-4de5-92cf-2f227426a398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.292205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1219/2463] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:45:40.292358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98205bf-a428-4de5-92cf-2f227426a398 admin admin] PATH_INFO: `/v3/projects/90bbc679faf847fe9ef79f7cea247d94/users/f8a6746337dc46dabd71eca90cadecc1/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.299912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323825-ce7f-4503-b28a-c95ef2302ec6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.300223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323825-ce7f-4503-b28a-c95ef2302ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.300513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87323825-ce7f-4503-b28a-c95ef2302ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.310278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98205bf-a428-4de5-92cf-2f227426a398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bbc679faf847fe9ef79f7cea247d94, user_id=f8a6746337dc46dabd71eca90cadecc1, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.313077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98205bf-a428-4de5-92cf-2f227426a398 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.321744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-87323825-ce7f-4503-b28a-c95ef2302ec6 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:40.322978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-87323825-ce7f-4503-b28a-c95ef2302ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:40.331104 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1220/2464] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:45:40.335394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87323825-ce7f-4503-b28a-c95ef2302ec6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.344995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87323825-ce7f-4503-b28a-c95ef2302ec6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.350491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.351185 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.351480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.351764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.352458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:45:40.352766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.354388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.371499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1221/2465] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:45:40.373568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.391531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97f38-9f7e-49cb-bc4c-f08f6cb341ea admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.393584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1245/2466] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:40 2022] PUT /identity/v3/projects/90bbc679faf847fe9ef79f7cea247d94/users/f8a6746337dc46dabd71eca90cadecc1/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:40.398718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98205bf-a428-4de5-92cf-2f227426a398 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.436944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:40.466478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60f099-8a6e-4c4a-bb30-41b6f0854903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.467243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60f099-8a6e-4c4a-bb30-41b6f0854903 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.467607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60f099-8a6e-4c4a-bb30-41b6f0854903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.467961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60f099-8a6e-4c4a-bb30-41b6f0854903 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.468708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60f099-8a6e-4c4a-bb30-41b6f0854903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.470690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60f099-8a6e-4c4a-bb30-41b6f0854903 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.491146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98205bf-a428-4de5-92cf-2f227426a398 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.496591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc63c6-dc19-449a-8cd6-720685a16230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.497199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc63c6-dc19-449a-8cd6-720685a16230 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.497477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc63c6-dc19-449a-8cd6-720685a16230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.497748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc63c6-dc19-449a-8cd6-720685a16230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.498402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc63c6-dc19-449a-8cd6-720685a16230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.500010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc63c6-dc19-449a-8cd6-720685a16230 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.513004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1246/2467] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:40.517734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc63c6-dc19-449a-8cd6-720685a16230 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.529129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc63c6-dc19-449a-8cd6-720685a16230 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.534099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6c524-57a3-4dea-922d-bca8e7329350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.534803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6c524-57a3-4dea-922d-bca8e7329350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.535089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6c524-57a3-4dea-922d-bca8e7329350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.536369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6c524-57a3-4dea-922d-bca8e7329350 admin admin] PATH_INFO: `/v3/projects/90bbc679faf847fe9ef79f7cea247d94/users/f8a6746337dc46dabd71eca90cadecc1/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.539489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6c524-57a3-4dea-922d-bca8e7329350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bbc679faf847fe9ef79f7cea247d94, user_id=f8a6746337dc46dabd71eca90cadecc1, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.543087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6c524-57a3-4dea-922d-bca8e7329350 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.555300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1247/2468] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:40 2022] PUT /identity/v3/projects/90bbc679faf847fe9ef79f7cea247d94/users/f8a6746337dc46dabd71eca90cadecc1/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:45:40.568784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2324-7399-4b7f-8414-68c1a170b69c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.569082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2324-7399-4b7f-8414-68c1a170b69c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.569394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2324-7399-4b7f-8414-68c1a170b69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.599135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-db5a2324-7399-4b7f-8414-68c1a170b69c None None] MFA Rules not processed for user `f8a6746337dc46dabd71eca90cadecc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:40.600304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-db5a2324-7399-4b7f-8414-68c1a170b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:40.627700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:40.663677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1222/2469] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:45:40 2022] POST /identity/v3/projects => generated 375 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:40.669613 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60f099-8a6e-4c4a-bb30-41b6f0854903 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.695922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1248/2470] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:45:40.711185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a2324-7399-4b7f-8414-68c1a170b69c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.713079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-db5a2324-7399-4b7f-8414-68c1a170b69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:40.792966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a2324-7399-4b7f-8414-68c1a170b69c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.811304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60f099-8a6e-4c4a-bb30-41b6f0854903 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.817140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862637d-c777-4879-964f-f4fe8f9b712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.817920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862637d-c777-4879-964f-f4fe8f9b712e admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.818236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862637d-c777-4879-964f-f4fe8f9b712e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.818512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862637d-c777-4879-964f-f4fe8f9b712e admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.819166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862637d-c777-4879-964f-f4fe8f9b712e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.821581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862637d-c777-4879-964f-f4fe8f9b712e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.856880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a522e0-0ef4-4e7b-8c41-80f2294c5c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.857579 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a522e0-0ef4-4e7b-8c41-80f2294c5c3d service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.857926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a522e0-0ef4-4e7b-8c41-80f2294c5c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.858266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a522e0-0ef4-4e7b-8c41-80f2294c5c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.927344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a522e0-0ef4-4e7b-8c41-80f2294c5c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.928832 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a522e0-0ef4-4e7b-8c41-80f2294c5c3d service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.939729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1249/2471] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:45:40.960131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1223/2472] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:45:40 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:40.964488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a522e0-0ef4-4e7b-8c41-80f2294c5c3d service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:40.975640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a522e0-0ef4-4e7b-8c41-80f2294c5c3d service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:40.980712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d86005-423d-4d3d-91da-5ef21c9671ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:40.981418 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d86005-423d-4d3d-91da-5ef21c9671ba admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:40.981724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d86005-423d-4d3d-91da-5ef21c9671ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:40.982004 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d86005-423d-4d3d-91da-5ef21c9671ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:40.982657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d86005-423d-4d3d-91da-5ef21c9671ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:40.984402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d86005-423d-4d3d-91da-5ef21c9671ba admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:40.996148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1250/2473] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:41.000404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862637d-c777-4879-964f-f4fe8f9b712e admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:41.011291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862637d-c777-4879-964f-f4fe8f9b712e admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:41.016741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c869a5-bbd4-47cb-a238-4c01e972e577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:41.017357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c869a5-bbd4-47cb-a238-4c01e972e577 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:41.017634 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c869a5-bbd4-47cb-a238-4c01e972e577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:41.017908 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c869a5-bbd4-47cb-a238-4c01e972e577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:41.018557 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c869a5-bbd4-47cb-a238-4c01e972e577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:41.020216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c869a5-bbd4-47cb-a238-4c01e972e577 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:41.032937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1224/2474] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:41.038415 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d86005-423d-4d3d-91da-5ef21c9671ba admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:41.049368 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d86005-423d-4d3d-91da-5ef21c9671ba admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:41.055596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f230c61-4b63-411e-bc5e-c20c67fecb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:41.056315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f230c61-4b63-411e-bc5e-c20c67fecb8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:41.056605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f230c61-4b63-411e-bc5e-c20c67fecb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:41.056886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f230c61-4b63-411e-bc5e-c20c67fecb8c admin admin] PATH_INFO: `/v3/projects/7e153c49233f415aabe4e6325b945ed6/users/8523e49705f44fe2b053ebf79d955cb1/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:41.073558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f230c61-4b63-411e-bc5e-c20c67fecb8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e153c49233f415aabe4e6325b945ed6, user_id=8523e49705f44fe2b053ebf79d955cb1, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:41.077345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f230c61-4b63-411e-bc5e-c20c67fecb8c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:41.079179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:41.163039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1251/2475] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:41 2022] PUT /identity/v3/projects/7e153c49233f415aabe4e6325b945ed6/users/8523e49705f44fe2b053ebf79d955cb1/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:45:41.174343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ccc15-5e76-447e-9939-154c15865efb None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:41.179119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ccc15-5e76-447e-9939-154c15865efb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:41.179119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ccc15-5e76-447e-9939-154c15865efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:41.209019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a58ccc15-5e76-447e-9939-154c15865efb None None] MFA Rules not processed for user `8523e49705f44fe2b053ebf79d955cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:41.209019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a58ccc15-5e76-447e-9939-154c15865efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:41.215966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f230c61-4b63-411e-bc5e-c20c67fecb8c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:41.276320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f230c61-4b63-411e-bc5e-c20c67fecb8c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:41.280643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:41.281249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:41.281533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:41.281834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:41.283444 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1225/2476] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:45:41.283747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:41.295617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ccc15-5e76-447e-9939-154c15865efb None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:41.297559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a58ccc15-5e76-447e-9939-154c15865efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:41.333187 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:41.358402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:41.359831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:41.370018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1252/2477] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:45:41.376891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ccc15-5e76-447e-9939-154c15865efb None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:41.384202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11a052-5bef-4aa9-ae9e-8f3fe58a4daa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:41.384844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11a052-5bef-4aa9-ae9e-8f3fe58a4daa service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:41.385125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11a052-5bef-4aa9-ae9e-8f3fe58a4daa service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:41.385398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11a052-5bef-4aa9-ae9e-8f3fe58a4daa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:41.393868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11a052-5bef-4aa9-ae9e-8f3fe58a4daa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:41.396107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11a052-5bef-4aa9-ae9e-8f3fe58a4daa service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:41.406074 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1226/2478] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:45:41.631518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:41.697633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2552335-0836-402d-928a-46704751f3a5 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:41.702239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:41.702870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:41.703160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:41.703500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] PATH_INFO: `/v3/users/8523e49705f44fe2b053ebf79d955cb1` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:41.704677 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8523e49705f44fe2b053ebf79d955cb1)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:41.706651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:41.892627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8523e49705f44fe2b053ebf79d955cb1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:41.893378 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8523e49705f44fe2b053ebf79d955cb1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:41.897080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=1,keystone:SELECT=92,keystone:DELETE=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:41.903672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8523e49705f44fe2b053ebf79d955cb1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:41.953690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8523e49705f44fe2b053ebf79d955cb1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:41.959696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8523e49705f44fe2b053ebf79d955cb1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:41.974616 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1253/2479] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:45:41 2022] DELETE /identity/v3/users/8523e49705f44fe2b053ebf79d955cb1 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:41.979657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296ba2-1183-4456-8596-2acae5ee9dfe None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:41.980209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296ba2-1183-4456-8596-2acae5ee9dfe None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:41.980465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296ba2-1183-4456-8596-2acae5ee9dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:42.002384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2e296ba2-1183-4456-8596-2acae5ee9dfe None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:42.003561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2e296ba2-1183-4456-8596-2acae5ee9dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:42.075060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1227/2480] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:42.085316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:42.086784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:42.277877 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2272-9a7a-4c2d-a3b3-bda362ce9222 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:42.282734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3a8ef-349c-45c0-b030-d7d0ddd9878c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:42.283335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a8ef-349c-45c0-b030-d7d0ddd9878c service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:42.283606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a8ef-349c-45c0-b030-d7d0ddd9878c service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:42.283874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a8ef-349c-45c0-b030-d7d0ddd9878c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:42.294171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3a8ef-349c-45c0-b030-d7d0ddd9878c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:42.295681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3a8ef-349c-45c0-b030-d7d0ddd9878c service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:42.306371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1254/2481] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:45:42.733010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296ba2-1183-4456-8596-2acae5ee9dfe None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:42.734725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2e296ba2-1183-4456-8596-2acae5ee9dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:42.751243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296ba2-1183-4456-8596-2acae5ee9dfe None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:42.755822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:42.756503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:42.756785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:42.757060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] PATH_INFO: `/v3/projects/7e153c49233f415aabe4e6325b945ed6` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:42.758839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e153c49233f415aabe4e6325b945ed6)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:42.760998 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:42.881887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e153c49233f415aabe4e6325b945ed6', 'request_id': 'req-cf595dfd-cac2-42f4-b732-3a69c0ba9171'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:42.887579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e153c49233f415aabe4e6325b945ed6', 'request_id': 'req-cf595dfd-cac2-42f4-b732-3a69c0ba9171'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:42.894246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e153c49233f415aabe4e6325b945ed6', 'request_id': 'req-cf595dfd-cac2-42f4-b732-3a69c0ba9171'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:42.971150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] The token cache is being invalidate because project 7e153c49233f415aabe4e6325b945ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:45:42.971508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:42.972348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cf595dfd-cac2-42f4-b732-3a69c0ba9171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:42.974596 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1228/2482] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:45:42 2022] DELETE /identity/v3/projects/7e153c49233f415aabe4e6325b945ed6 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:42.980095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345b57-4555-421a-bf90-4a9ae6e81042 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:42.980722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345b57-4555-421a-bf90-4a9ae6e81042 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:42.981003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8345b57-4555-421a-bf90-4a9ae6e81042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.007896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-a8345b57-4555-421a-bf90-4a9ae6e81042 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:43.009438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a8345b57-4555-421a-bf90-4a9ae6e81042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:43.080046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1255/2483] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:43.089090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df0215-cf15-404e-b2db-2405fd763030 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:43.089623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df0215-cf15-404e-b2db-2405fd763030 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:43.089719 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df0215-cf15-404e-b2db-2405fd763030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.109933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d9df0215-cf15-404e-b2db-2405fd763030 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:43.111190 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d9df0215-cf15-404e-b2db-2405fd763030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:43.119743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1229/2484] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:43.124016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345b57-4555-421a-bf90-4a9ae6e81042 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:43.135015 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8345b57-4555-421a-bf90-4a9ae6e81042 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:43.139750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:43.141843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:43.141843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:43.141843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.141843 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:45:43.142179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:43.143528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:43.156587 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1256/2485] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:45:43.161390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df0215-cf15-404e-b2db-2405fd763030 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:43.170928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df0215-cf15-404e-b2db-2405fd763030 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:43.175612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325e84-5dea-4791-8e65-ddde5969e9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:43.176554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325e84-5dea-4791-8e65-ddde5969e9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:43.176837 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325e84-5dea-4791-8e65-ddde5969e9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:43.177111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96325e84-5dea-4791-8e65-ddde5969e9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.177748 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96325e84-5dea-4791-8e65-ddde5969e9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:43.179651 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96325e84-5dea-4791-8e65-ddde5969e9a5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:43.274016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1230/2486] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:45:43 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:43.277959 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:43.290338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e92513-2ec4-4c0a-ad19-2ac9dd134007 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:43.364321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:43.370868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989ce86-a71e-449c-a2d6-e4d8860e839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:43.371486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989ce86-a71e-449c-a2d6-e4d8860e839f admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:43.371786 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989ce86-a71e-449c-a2d6-e4d8860e839f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:43.372055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989ce86-a71e-449c-a2d6-e4d8860e839f admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.372860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989ce86-a71e-449c-a2d6-e4d8860e839f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:43.374834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989ce86-a71e-449c-a2d6-e4d8860e839f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:43.558525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1257/2487] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:45:43 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:43.564384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325e84-5dea-4791-8e65-ddde5969e9a5 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:43.582618 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96325e84-5dea-4791-8e65-ddde5969e9a5 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:43.588301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90d10d-6bcb-41f1-a0d7-ea3d80b3ed66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:43.598887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90d10d-6bcb-41f1-a0d7-ea3d80b3ed66 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:43.598887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90d10d-6bcb-41f1-a0d7-ea3d80b3ed66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:43.598887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90d10d-6bcb-41f1-a0d7-ea3d80b3ed66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.598887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90d10d-6bcb-41f1-a0d7-ea3d80b3ed66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:43.598887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90d10d-6bcb-41f1-a0d7-ea3d80b3ed66 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:43.598887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:43.603533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1231/2488] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:43.607424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989ce86-a71e-449c-a2d6-e4d8860e839f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:43.620224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989ce86-a71e-449c-a2d6-e4d8860e839f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:43.625295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21ce00-7470-486a-9f4c-99984dfe89a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:43.627442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21ce00-7470-486a-9f4c-99984dfe89a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:43.627442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21ce00-7470-486a-9f4c-99984dfe89a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:43.627442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21ce00-7470-486a-9f4c-99984dfe89a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.627442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21ce00-7470-486a-9f4c-99984dfe89a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:43.629662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21ce00-7470-486a-9f4c-99984dfe89a5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:43.640707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1258/2489] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:43.645572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90d10d-6bcb-41f1-a0d7-ea3d80b3ed66 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:43.656439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90d10d-6bcb-41f1-a0d7-ea3d80b3ed66 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:43.661904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e04fe-19b9-4e3f-9ec0-6d4dc61fa330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:43.662562 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e04fe-19b9-4e3f-9ec0-6d4dc61fa330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:43.662845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e04fe-19b9-4e3f-9ec0-6d4dc61fa330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:43.663120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e04fe-19b9-4e3f-9ec0-6d4dc61fa330 admin admin] PATH_INFO: `/v3/projects/919356c1ba6a467083fd5a11de138f01/users/8ed3ddf466f14bd7956c081d1f64f17e/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.681607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e04fe-19b9-4e3f-9ec0-6d4dc61fa330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=919356c1ba6a467083fd5a11de138f01, user_id=8ed3ddf466f14bd7956c081d1f64f17e, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:43.685031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e04fe-19b9-4e3f-9ec0-6d4dc61fa330 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:43.818299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1232/2490] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:43 2022] PUT /identity/v3/projects/919356c1ba6a467083fd5a11de138f01/users/8ed3ddf466f14bd7956c081d1f64f17e/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:43.828625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bce3-d8ca-4fdc-9081-f1c02ef6dbb8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:43.828914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bce3-d8ca-4fdc-9081-f1c02ef6dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:43.832891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bce3-d8ca-4fdc-9081-f1c02ef6dbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:43.864131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5a85bce3-d8ca-4fdc-9081-f1c02ef6dbb8 None None] MFA Rules not processed for user `8ed3ddf466f14bd7956c081d1f64f17e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:43.865509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5a85bce3-d8ca-4fdc-9081-f1c02ef6dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:43.950141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1259/2491] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:45:43.954930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e04fe-19b9-4e3f-9ec0-6d4dc61fa330 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:44.023408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e04fe-19b9-4e3f-9ec0-6d4dc61fa330 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:44.028461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962764e7-035a-4acc-9572-7159fb6d12ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:44.029095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962764e7-035a-4acc-9572-7159fb6d12ff admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:44.029373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962764e7-035a-4acc-9572-7159fb6d12ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:44.029642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962764e7-035a-4acc-9572-7159fb6d12ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:44.030279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962764e7-035a-4acc-9572-7159fb6d12ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:44.032251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962764e7-035a-4acc-9572-7159fb6d12ff admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:44.148944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1233/2492] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:45:43 2022] POST /identity/v3/projects => generated 365 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:44.153735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85bce3-d8ca-4fdc-9081-f1c02ef6dbb8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:44.168805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85bce3-d8ca-4fdc-9081-f1c02ef6dbb8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:44.259785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c31b47-718a-4a57-93e5-c8ccf7c38f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:44.266116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c31b47-718a-4a57-93e5-c8ccf7c38f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:44.266116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c31b47-718a-4a57-93e5-c8ccf7c38f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:44.266116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c31b47-718a-4a57-93e5-c8ccf7c38f15 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:44.266116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c31b47-718a-4a57-93e5-c8ccf7c38f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:44.266116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c31b47-718a-4a57-93e5-c8ccf7c38f15 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:44.408244 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1260/2493] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:45:44 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:44.409856 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962764e7-035a-4acc-9572-7159fb6d12ff admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:44.425430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962764e7-035a-4acc-9572-7159fb6d12ff admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:44.432528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3bca-f6f7-47e0-b149-08a6f975c8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:44.433233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3bca-f6f7-47e0-b149-08a6f975c8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:44.433591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3bca-f6f7-47e0-b149-08a6f975c8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:44.433980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac3bca-f6f7-47e0-b149-08a6f975c8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:44.434810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3bca-f6f7-47e0-b149-08a6f975c8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:44.436910 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac3bca-f6f7-47e0-b149-08a6f975c8c6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:44.450523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1234/2494] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:44.455019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c31b47-718a-4a57-93e5-c8ccf7c38f15 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:44.465886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c31b47-718a-4a57-93e5-c8ccf7c38f15 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:44.471013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9d273-01cb-4cc7-9e3d-297f3fb562e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:44.471692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9d273-01cb-4cc7-9e3d-297f3fb562e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:44.472274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9d273-01cb-4cc7-9e3d-297f3fb562e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:44.472489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9d273-01cb-4cc7-9e3d-297f3fb562e7 admin admin] PATH_INFO: `/v3/projects/c877faa32fcf48978c90186fe5d962a8/users/a46ab924bf45447d88a4b573207c7448/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:44.492551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9d273-01cb-4cc7-9e3d-297f3fb562e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c877faa32fcf48978c90186fe5d962a8, user_id=a46ab924bf45447d88a4b573207c7448, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:44.495322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9d273-01cb-4cc7-9e3d-297f3fb562e7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:44.578866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1261/2495] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:44 2022] PUT /identity/v3/projects/c877faa32fcf48978c90186fe5d962a8/users/a46ab924bf45447d88a4b573207c7448/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:44.583974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3bca-f6f7-47e0-b149-08a6f975c8c6 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:44.595649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac3bca-f6f7-47e0-b149-08a6f975c8c6 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:44.657930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4e885-4c2f-43e0-bceb-d0a094625268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:44.658546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4e885-4c2f-43e0-bceb-d0a094625268 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:44.658830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4e885-4c2f-43e0-bceb-d0a094625268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:44.659105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4e885-4c2f-43e0-bceb-d0a094625268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:44.659757 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4e885-4c2f-43e0-bceb-d0a094625268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:44.662075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4e885-4c2f-43e0-bceb-d0a094625268 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:44.672626 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1235/2496] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:45:44.677956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9d273-01cb-4cc7-9e3d-297f3fb562e7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:44.688520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9d273-01cb-4cc7-9e3d-297f3fb562e7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:44.694421 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:44.695504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:44.700916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:44.700916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] PATH_INFO: `/v3/projects/c877faa32fcf48978c90186fe5d962a8/users/a46ab924bf45447d88a4b573207c7448/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:44.700916 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c877faa32fcf48978c90186fe5d962a8, user_id=a46ab924bf45447d88a4b573207c7448, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:44.701725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:44.716229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1262/2497] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:44 2022] PUT /identity/v3/projects/c877faa32fcf48978c90186fe5d962a8/users/a46ab924bf45447d88a4b573207c7448/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:44.729886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149299a4-754f-4439-8362-3c4615f3ecdc None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:44.730172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149299a4-754f-4439-8362-3c4615f3ecdc None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:44.730453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149299a4-754f-4439-8362-3c4615f3ecdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:44.762143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-149299a4-754f-4439-8362-3c4615f3ecdc None None] MFA Rules not processed for user `a46ab924bf45447d88a4b573207c7448`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:44.763606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-149299a4-754f-4439-8362-3c4615f3ecdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:44.800420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:44.854886 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1236/2498] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:44.891731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:44.893597 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:44.925689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:44.966653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe1cb7-e7e3-4714-9669-60e12da54f1a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:44.970220 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f65ca-62b3-40fd-9ff3-43fa45d64f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:44.970876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f65ca-62b3-40fd-9ff3-43fa45d64f3b service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:44.971152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f65ca-62b3-40fd-9ff3-43fa45d64f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:44.971419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f65ca-62b3-40fd-9ff3-43fa45d64f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:44.981872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f65ca-62b3-40fd-9ff3-43fa45d64f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:44.983292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f65ca-62b3-40fd-9ff3-43fa45d64f3b service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:44.994441 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1263/2499] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:45:46.159578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149299a4-754f-4439-8362-3c4615f3ecdc None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:46.226717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149299a4-754f-4439-8362-3c4615f3ecdc None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:46.231408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:46.232066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:46.232375 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:46.232669 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] PATH_INFO: `/v3/users/8ed3ddf466f14bd7956c081d1f64f17e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:46.233826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ed3ddf466f14bd7956c081d1f64f17e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:46.235797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:46.471913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ed3ddf466f14bd7956c081d1f64f17e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:46.473467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ed3ddf466f14bd7956c081d1f64f17e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:46.481852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ed3ddf466f14bd7956c081d1f64f17e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:46.483524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ed3ddf466f14bd7956c081d1f64f17e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:46.571036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ed3ddf466f14bd7956c081d1f64f17e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:46.594544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1237/2500] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:45:46 2022] DELETE /identity/v3/users/8ed3ddf466f14bd7956c081d1f64f17e => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:46.602442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d60677-9b52-41f6-b6a3-60d213f4de14 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:46.602442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d60677-9b52-41f6-b6a3-60d213f4de14 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:46.602442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d60677-9b52-41f6-b6a3-60d213f4de14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:46.621126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-76d60677-9b52-41f6-b6a3-60d213f4de14 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:46.622437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-76d60677-9b52-41f6-b6a3-60d213f4de14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:46.687930 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1264/2501] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:45:46.699427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:46.701842 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:46.765844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:46.782589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de08e0-2540-4463-8e25-a316c2b5365c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:46.782589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42760dea-1179-4781-818c-812b658cc2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:46.782589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42760dea-1179-4781-818c-812b658cc2db service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:46.782589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42760dea-1179-4781-818c-812b658cc2db service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:46.783021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42760dea-1179-4781-818c-812b658cc2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:46.784697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42760dea-1179-4781-818c-812b658cc2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:46.786940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42760dea-1179-4781-818c-812b658cc2db service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:46.797372 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1238/2502] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:45:47.252531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d60677-9b52-41f6-b6a3-60d213f4de14 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:47.253976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-76d60677-9b52-41f6-b6a3-60d213f4de14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:47.265755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d60677-9b52-41f6-b6a3-60d213f4de14 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:47.271382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:47.272071 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:47.272385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:47.272660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] PATH_INFO: `/v3/projects/919356c1ba6a467083fd5a11de138f01` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:47.273797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=919356c1ba6a467083fd5a11de138f01)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:47.275692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:47.370387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '919356c1ba6a467083fd5a11de138f01', 'request_id': 'req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.375820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '919356c1ba6a467083fd5a11de138f01', 'request_id': 'req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.439524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '919356c1ba6a467083fd5a11de138f01', 'request_id': 'req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.451246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] The token cache is being invalidate because project 919356c1ba6a467083fd5a11de138f01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:45:47.451528 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.452147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.453656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1265/2503] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:45:47 2022] DELETE /identity/v3/projects/919356c1ba6a467083fd5a11de138f01 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:47.458354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42760dea-1179-4781-818c-812b658cc2db service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:47.468227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42760dea-1179-4781-818c-812b658cc2db service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:47.528213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:47.528855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:47.529139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:47.529416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] PATH_INFO: `/v3/users/a46ab924bf45447d88a4b573207c7448` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:47.530563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a46ab924bf45447d88a4b573207c7448)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:47.532593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:47.726544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a46ab924bf45447d88a4b573207c7448'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.727301 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a46ab924bf45447d88a4b573207c7448'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.738479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a46ab924bf45447d88a4b573207c7448'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.739571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a46ab924bf45447d88a4b573207c7448'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.822379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4b722ed7-0559-4987-b657-949ab7947b0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a46ab924bf45447d88a4b573207c7448'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:47.843501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1239/2504] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:45:47 2022] DELETE /identity/v3/users/a46ab924bf45447d88a4b573207c7448 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:48.269481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:48.271028 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:48.285476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb1ae1-b3ed-4acb-99b5-619acb38b5b0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:48.332153 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:48.348789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:48.349527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:48.349856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:48.350181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] PATH_INFO: `/v3/projects/c877faa32fcf48978c90186fe5d962a8` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:48.351478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c877faa32fcf48978c90186fe5d962a8)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:48.353500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:48.482704 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c877faa32fcf48978c90186fe5d962a8', 'request_id': 'req-e9393e83-2a22-4009-8ba7-32c25851e2c3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:48.489075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c877faa32fcf48978c90186fe5d962a8', 'request_id': 'req-e9393e83-2a22-4009-8ba7-32c25851e2c3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:48.582187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c877faa32fcf48978c90186fe5d962a8', 'request_id': 'req-e9393e83-2a22-4009-8ba7-32c25851e2c3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:48.595032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] The token cache is being invalidate because project c877faa32fcf48978c90186fe5d962a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:45:48.595514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:48.596216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e9393e83-2a22-4009-8ba7-32c25851e2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:45:48.598406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1266/2505] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:45:48 2022] DELETE /identity/v3/projects/c877faa32fcf48978c90186fe5d962a8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:45:48.603405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bef2b-7509-440f-88cb-0fe41fc97088 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:48.603912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bef2b-7509-440f-88cb-0fe41fc97088 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:48.604027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bef2b-7509-440f-88cb-0fe41fc97088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:48.625249 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e62bef2b-7509-440f-88cb-0fe41fc97088 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:48.626500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e62bef2b-7509-440f-88cb-0fe41fc97088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:48.687073 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1240/2506] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:48.695373 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cc8ed-8b27-4610-85ae-e2e696ba3845 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:48.695673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cc8ed-8b27-4610-85ae-e2e696ba3845 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:48.695950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cc8ed-8b27-4610-85ae-e2e696ba3845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:48.717526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-144cc8ed-8b27-4610-85ae-e2e696ba3845 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:48.718723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-144cc8ed-8b27-4610-85ae-e2e696ba3845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:48.727590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1267/2507] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:48.732105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bef2b-7509-440f-88cb-0fe41fc97088 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:48.744924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bef2b-7509-440f-88cb-0fe41fc97088 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:48.747297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:48.747912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:48.748228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:48.748636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:48.749302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:45:48.749607 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:48.751285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:48.759031 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:48.764324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1241/2508] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:45:48.769038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144cc8ed-8b27-4610-85ae-e2e696ba3845 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:48.779653 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144cc8ed-8b27-4610-85ae-e2e696ba3845 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:48.784592 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49038b-2b90-48c4-abad-415c4d737d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:48.785784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49038b-2b90-48c4-abad-415c4d737d6c admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:48.786222 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49038b-2b90-48c4-abad-415c4d737d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:48.786494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49038b-2b90-48c4-abad-415c4d737d6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:48.787151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49038b-2b90-48c4-abad-415c4d737d6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:48.789107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49038b-2b90-48c4-abad-415c4d737d6c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:48.895816 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1268/2509] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:45:48 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:48.897021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:48.907675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89748a6c-fe14-4fbb-9a14-d771a4293a09 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:48.987282 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b512c6c-acd7-4a41-b8e0-2e96646a8cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:48.988041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b512c6c-acd7-4a41-b8e0-2e96646a8cef admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:48.989000 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b512c6c-acd7-4a41-b8e0-2e96646a8cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:48.989344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b512c6c-acd7-4a41-b8e0-2e96646a8cef admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:48.990208 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b512c6c-acd7-4a41-b8e0-2e96646a8cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:48.992174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b512c6c-acd7-4a41-b8e0-2e96646a8cef admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:49.123984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1242/2510] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:45:48 2022] POST /identity/v3/users => generated 282 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:45:49.128108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49038b-2b90-48c4-abad-415c4d737d6c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:49.140097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49038b-2b90-48c4-abad-415c4d737d6c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:49.145880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf739fc-5900-4850-848e-22d206fb27de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:49.146533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf739fc-5900-4850-848e-22d206fb27de admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:49.146810 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf739fc-5900-4850-848e-22d206fb27de admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:49.147081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf739fc-5900-4850-848e-22d206fb27de admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:49.147737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf739fc-5900-4850-848e-22d206fb27de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:49.149409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf739fc-5900-4850-848e-22d206fb27de admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:49.162545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1269/2511] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:49.167653 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b512c6c-acd7-4a41-b8e0-2e96646a8cef admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:49.179544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b512c6c-acd7-4a41-b8e0-2e96646a8cef admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:49.184561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc815c6-0ed4-4b7e-a852-c4ba8d9d2c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:49.185176 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc815c6-0ed4-4b7e-a852-c4ba8d9d2c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:49.185461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc815c6-0ed4-4b7e-a852-c4ba8d9d2c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:49.185778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc815c6-0ed4-4b7e-a852-c4ba8d9d2c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:49.188951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc815c6-0ed4-4b7e-a852-c4ba8d9d2c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:49.190955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc815c6-0ed4-4b7e-a852-c4ba8d9d2c25 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:49.202018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1243/2512] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:45:49.206234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf739fc-5900-4850-848e-22d206fb27de admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:49.217872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf739fc-5900-4850-848e-22d206fb27de admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:49.223868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b32d6-3822-4bc9-a0bc-663183343d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:49.224621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b32d6-3822-4bc9-a0bc-663183343d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:49.224900 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b32d6-3822-4bc9-a0bc-663183343d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:49.225191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b32d6-3822-4bc9-a0bc-663183343d9f admin admin] PATH_INFO: `/v3/projects/789e0658710b4aa082d8311138d33cf9/users/55dc6320d0d04f45bd364e2f67a1f15e/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:49.242414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b32d6-3822-4bc9-a0bc-663183343d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=789e0658710b4aa082d8311138d33cf9, user_id=55dc6320d0d04f45bd364e2f67a1f15e, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:49.245409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b32d6-3822-4bc9-a0bc-663183343d9f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:49.330642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1270/2513] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:49 2022] PUT /identity/v3/projects/789e0658710b4aa082d8311138d33cf9/users/55dc6320d0d04f45bd364e2f67a1f15e/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:49.346539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40196c7b-d087-4e2b-be54-f63128e9e261 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:49.346825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40196c7b-d087-4e2b-be54-f63128e9e261 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:49.347175 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40196c7b-d087-4e2b-be54-f63128e9e261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:49.378166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-40196c7b-d087-4e2b-be54-f63128e9e261 None None] MFA Rules not processed for user `55dc6320d0d04f45bd364e2f67a1f15e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:49.379451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-40196c7b-d087-4e2b-be54-f63128e9e261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:49.456107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1244/2514] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:49.461556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b32d6-3822-4bc9-a0bc-663183343d9f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:49.530072 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b32d6-3822-4bc9-a0bc-663183343d9f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:49.536053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5e69c-6dcf-460c-bc02-9e31adc172cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:49.536757 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5e69c-6dcf-460c-bc02-9e31adc172cb admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:49.537037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5e69c-6dcf-460c-bc02-9e31adc172cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:49.537308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5e69c-6dcf-460c-bc02-9e31adc172cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:49.537956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5e69c-6dcf-460c-bc02-9e31adc172cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:49.539923 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5e69c-6dcf-460c-bc02-9e31adc172cb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:49.556914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:49.670053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1271/2515] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:45:49 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:45:49.674309 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40196c7b-d087-4e2b-be54-f63128e9e261 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:49.685752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40196c7b-d087-4e2b-be54-f63128e9e261 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:49.747531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e308f-ec5e-432c-ac35-0f0821c98dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:49.748169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e308f-ec5e-432c-ac35-0f0821c98dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:49.748482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e308f-ec5e-432c-ac35-0f0821c98dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:49.748760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e308f-ec5e-432c-ac35-0f0821c98dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:49.749426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e308f-ec5e-432c-ac35-0f0821c98dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:49.751392 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e308f-ec5e-432c-ac35-0f0821c98dc0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:49.762990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:45:49.886724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1245/2516] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:45:49 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:45:49.895088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5e69c-6dcf-460c-bc02-9e31adc172cb admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:49.909973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5e69c-6dcf-460c-bc02-9e31adc172cb admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:49.915324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0b61e-9340-4098-b565-6507cd78bee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:49.916000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0b61e-9340-4098-b565-6507cd78bee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:49.916322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0b61e-9340-4098-b565-6507cd78bee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:49.916607 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0b61e-9340-4098-b565-6507cd78bee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:49.917371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0b61e-9340-4098-b565-6507cd78bee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:49.919023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0b61e-9340-4098-b565-6507cd78bee5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:49.932124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1272/2517] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:49.935272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e308f-ec5e-432c-ac35-0f0821c98dc0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:49.946126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e308f-ec5e-432c-ac35-0f0821c98dc0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:49.950933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe2ae2-aaaa-4a16-be1c-11583c9b363c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:49.951586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2ae2-aaaa-4a16-be1c-11583c9b363c admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:49.951872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2ae2-aaaa-4a16-be1c-11583c9b363c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:49.952144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2ae2-aaaa-4a16-be1c-11583c9b363c admin admin] PATH_INFO: `/v3/projects/5898c58b80dd4a0db6d16f31edd1348f/users/4a9e26e079bf468c82beea2d5785931f/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:49.968111 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe2ae2-aaaa-4a16-be1c-11583c9b363c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5898c58b80dd4a0db6d16f31edd1348f, user_id=4a9e26e079bf468c82beea2d5785931f, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:49.970848 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe2ae2-aaaa-4a16-be1c-11583c9b363c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:50.091678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1246/2518] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:49 2022] PUT /identity/v3/projects/5898c58b80dd4a0db6d16f31edd1348f/users/4a9e26e079bf468c82beea2d5785931f/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:45:50.096798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0b61e-9340-4098-b565-6507cd78bee5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:50.108769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0b61e-9340-4098-b565-6507cd78bee5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:50.170666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac468027-2782-4a2f-aa1b-0c01d76d86eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:50.171273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468027-2782-4a2f-aa1b-0c01d76d86eb admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:50.171552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468027-2782-4a2f-aa1b-0c01d76d86eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:50.171826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac468027-2782-4a2f-aa1b-0c01d76d86eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:50.172608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac468027-2782-4a2f-aa1b-0c01d76d86eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:50.174178 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac468027-2782-4a2f-aa1b-0c01d76d86eb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:50.185029 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1273/2519] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:45:50.189572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe2ae2-aaaa-4a16-be1c-11583c9b363c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:50.201350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe2ae2-aaaa-4a16-be1c-11583c9b363c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:50.206459 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:50.207314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:50.207600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:50.208046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] PATH_INFO: `/v3/projects/5898c58b80dd4a0db6d16f31edd1348f/users/4a9e26e079bf468c82beea2d5785931f/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:50.210194 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5898c58b80dd4a0db6d16f31edd1348f, user_id=4a9e26e079bf468c82beea2d5785931f, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:50.212851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:50.227470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1247/2520] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:45:50 2022] PUT /identity/v3/projects/5898c58b80dd4a0db6d16f31edd1348f/users/4a9e26e079bf468c82beea2d5785931f/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:45:50.246020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc3429-cb29-4534-b200-4ea9d9b43d19 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:50.247067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc3429-cb29-4534-b200-4ea9d9b43d19 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:50.247067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc3429-cb29-4534-b200-4ea9d9b43d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:50.283589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d0bc3429-cb29-4534-b200-4ea9d9b43d19 None None] MFA Rules not processed for user `4a9e26e079bf468c82beea2d5785931f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:45:50.285090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d0bc3429-cb29-4534-b200-4ea9d9b43d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:50.380051 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1274/2521] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:45:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:45:50.386977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:50.388987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:45:50.459592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f4ea9-4514-40ad-851b-6d259d398d18 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:50.463463 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:50.464094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:50.464413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:50.464689 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:50.478029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:50.480551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:50.493397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1248/2522] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:45:50.781083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc3429-cb29-4534-b200-4ea9d9b43d19 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:45:50.793485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc3429-cb29-4534-b200-4ea9d9b43d19 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:45:50.798128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f639ec-1d04-4e45-b28b-e038fcda1d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:45:50.799312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f639ec-1d04-4e45-b28b-e038fcda1d30 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:45:50.799312 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f639ec-1d04-4e45-b28b-e038fcda1d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:45:50.799541 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f639ec-1d04-4e45-b28b-e038fcda1d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:45:50.863356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f639ec-1d04-4e45-b28b-e038fcda1d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:45:50.865020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f639ec-1d04-4e45-b28b-e038fcda1d30 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:45:50.874714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1275/2523] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:45:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:00.487868 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:00.870345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=98 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:01.307120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:01.308724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:01.370369 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fce3d8-9378-417e-84a2-6aa09e66718e service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:01.374416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7367-c2ea-46dd-b984-0730342c7032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:01.375044 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7367-c2ea-46dd-b984-0730342c7032 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:01.375328 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7367-c2ea-46dd-b984-0730342c7032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:01.375603 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7367-c2ea-46dd-b984-0730342c7032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:01.377658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b19b7367-c2ea-46dd-b984-0730342c7032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:01.458478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7367-c2ea-46dd-b984-0730342c7032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:01.460479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7367-c2ea-46dd-b984-0730342c7032 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:01.471237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1249/2524] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:06.020321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f639ec-1d04-4e45-b28b-e038fcda1d30 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:06.047152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f639ec-1d04-4e45-b28b-e038fcda1d30 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:06.052888 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:06.053856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:06.054317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:06.054942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] PATH_INFO: `/v3/users/55dc6320d0d04f45bd364e2f67a1f15e` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:06.056598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dc6320d0d04f45bd364e2f67a1f15e)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:06.060392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:06.262175 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dc6320d0d04f45bd364e2f67a1f15e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:06.262850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dc6320d0d04f45bd364e2f67a1f15e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:06.272793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dc6320d0d04f45bd364e2f67a1f15e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:06.341226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dc6320d0d04f45bd364e2f67a1f15e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:06.341903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dc6320d0d04f45bd364e2f67a1f15e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:06.361405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1276/2525] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:06 2022] DELETE /identity/v3/users/55dc6320d0d04f45bd364e2f67a1f15e => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:06.366035 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26f693-b1c8-439a-b30d-8c476d5d1e2d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:06.366540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26f693-b1c8-439a-b30d-8c476d5d1e2d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:06.366644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26f693-b1c8-439a-b30d-8c476d5d1e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:06.394711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6d26f693-b1c8-439a-b30d-8c476d5d1e2d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:06.395958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6d26f693-b1c8-439a-b30d-8c476d5d1e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:06.513778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1250/2526] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:06.526779 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:06.528339 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:06.590895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc83631-e0c0-4b11-ac13-0b56b44b458c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:06.595595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:06.597093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:06.597093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:06.597093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:06.611021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:06.612400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:06.623454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1277/2527] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:06.624676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26f693-b1c8-439a-b30d-8c476d5d1e2d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:06.626697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6d26f693-b1c8-439a-b30d-8c476d5d1e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:06.689525 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:06.708915 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26f693-b1c8-439a-b30d-8c476d5d1e2d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:06.714248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:06.714865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:06.715148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:06.715431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:06.717240 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:06.730455 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:06.735011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:06.745297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1251/2528] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:06.948130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:06.950621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:06.966673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732be0c7-8faf-49f4-aa7d-6527a48e9e18 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:06.971988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:06.972650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:06.973049 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:06.973406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] PATH_INFO: `/v3/users/773dc87736034257b3d1ae2e8f02bf23` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:06.975221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] RBAC: Authorizing `identity:delete_user(user_id=773dc87736034257b3d1ae2e8f02bf23)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:06.977200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:07.196435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '773dc87736034257b3d1ae2e8f02bf23'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.197286 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '773dc87736034257b3d1ae2e8f02bf23'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.205420 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '773dc87736034257b3d1ae2e8f02bf23'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.269108 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:07.270801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:07.291979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28753c-f5f1-4682-b0bd-eb52b83d6b9b service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:07.296434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '773dc87736034257b3d1ae2e8f02bf23'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.297258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '773dc87736034257b3d1ae2e8f02bf23'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.298942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:07.299809 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:07.300313 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:07.300775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] PATH_INFO: `/v3/projects/789e0658710b4aa082d8311138d33cf9` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:07.303084 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=789e0658710b4aa082d8311138d33cf9)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:07.307436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:07.307642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:07.325627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1278/2529] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:06 2022] DELETE /identity/v3/users/773dc87736034257b3d1ae2e8f02bf23 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:07.436889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789e0658710b4aa082d8311138d33cf9', 'request_id': 'req-021abe45-f792-4134-9297-0983d357a8a3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.442632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789e0658710b4aa082d8311138d33cf9', 'request_id': 'req-021abe45-f792-4134-9297-0983d357a8a3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.449422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789e0658710b4aa082d8311138d33cf9', 'request_id': 'req-021abe45-f792-4134-9297-0983d357a8a3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.547767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] The token cache is being invalidate because project 789e0658710b4aa082d8311138d33cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:07.548060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.549262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.550433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1252/2530] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:07 2022] DELETE /identity/v3/projects/789e0658710b4aa082d8311138d33cf9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:07.557082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:07.558787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:07.589782 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:07.592090 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:07.647760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd6640-fd56-438f-95ec-b5a65e2e790c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:07.697995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021abe45-f792-4134-9297-0983d357a8a3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:07.700408 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:07.701054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:07.701337 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:07.701614 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] PATH_INFO: `/v3/projects/4a3794670e8d4c9985547d557d6969e0` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:07.702787 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a3794670e8d4c9985547d557d6969e0)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:07.704878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:07.719046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:07.719838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:07.720205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:07.720829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] PATH_INFO: `/v3/users/4a9e26e079bf468c82beea2d5785931f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:07.722360 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a9e26e079bf468c82beea2d5785931f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:07.725358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:07.897840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a3794670e8d4c9985547d557d6969e0', 'request_id': 'req-c47e54e4-afc2-497d-91cb-6e4ba1cea499'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.898529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a9e26e079bf468c82beea2d5785931f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.899380 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a9e26e079bf468c82beea2d5785931f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.903819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a3794670e8d4c9985547d557d6969e0', 'request_id': 'req-c47e54e4-afc2-497d-91cb-6e4ba1cea499'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.907171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a9e26e079bf468c82beea2d5785931f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.910446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a3794670e8d4c9985547d557d6969e0', 'request_id': 'req-c47e54e4-afc2-497d-91cb-6e4ba1cea499'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:07.913551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:08.010392 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a9e26e079bf468c82beea2d5785931f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:08.011033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-dcb9eb27-4f53-4817-99e8-e6f73c732b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a9e26e079bf468c82beea2d5785931f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:08.030128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1279/2531] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:07 2022] DELETE /identity/v3/users/4a9e26e079bf468c82beea2d5785931f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:08.095681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] The token cache is being invalidate because project 4a3794670e8d4c9985547d557d6969e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:08.096064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:08.096775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c47e54e4-afc2-497d-91cb-6e4ba1cea499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:08.098366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1253/2532] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:07 2022] DELETE /identity/v3/projects/4a3794670e8d4c9985547d557d6969e0 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:08.103495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb07be-c32f-4a85-ac06-48ae0133834a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.103784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb07be-c32f-4a85-ac06-48ae0133834a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.104062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb07be-c32f-4a85-ac06-48ae0133834a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.124767 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-22cb07be-c32f-4a85-ac06-48ae0133834a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:08.125995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-22cb07be-c32f-4a85-ac06-48ae0133834a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:08.196116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1280/2533] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:08.203937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42ed9-4339-4135-beac-de5a244e6855 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.204317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42ed9-4339-4135-beac-de5a244e6855 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.204598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42ed9-4339-4135-beac-de5a244e6855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.224484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fcc42ed9-4339-4135-beac-de5a244e6855 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:08.225840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fcc42ed9-4339-4135-beac-de5a244e6855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:08.233881 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1254/2534] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:08.238229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb07be-c32f-4a85-ac06-48ae0133834a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:08.249076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb07be-c32f-4a85-ac06-48ae0133834a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:08.253693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:08.254343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.254633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.254907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.256343 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:46:08.256714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:08.258457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:08.270825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1281/2535] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:46:08.275501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc42ed9-4339-4135-beac-de5a244e6855 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:08.285231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc42ed9-4339-4135-beac-de5a244e6855 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:08.292838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:08.292838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.292838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.292838 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.293242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:08.294009 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:08.458099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1255/2536] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:08 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:08.462387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:08.472911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4978b-9397-4ad1-8244-254d065834a9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:08.530898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041e59c-74f4-4055-82e9-b106d740f48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:08.531504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e59c-74f4-4055-82e9-b106d740f48b admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.531788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e59c-74f4-4055-82e9-b106d740f48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.532067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e59c-74f4-4055-82e9-b106d740f48b admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.532734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041e59c-74f4-4055-82e9-b106d740f48b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:08.534774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1041e59c-74f4-4055-82e9-b106d740f48b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:08.625158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:08.628286 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:08.647407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf95dea-5d94-4185-a853-d9720cdffa97 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:08.649770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:08.650418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.650767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.650972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] PATH_INFO: `/v3/projects/5898c58b80dd4a0db6d16f31edd1348f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.652153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5898c58b80dd4a0db6d16f31edd1348f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:08.654087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:08.726057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1282/2537] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:08 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:08.730266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041e59c-74f4-4055-82e9-b106d740f48b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:08.740881 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041e59c-74f4-4055-82e9-b106d740f48b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:08.748718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b948bb-3b43-45ef-8daf-662cc61fa54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:08.748718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b948bb-3b43-45ef-8daf-662cc61fa54f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.748718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b948bb-3b43-45ef-8daf-662cc61fa54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.748718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b948bb-3b43-45ef-8daf-662cc61fa54f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.749408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b948bb-3b43-45ef-8daf-662cc61fa54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:08.749778 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b948bb-3b43-45ef-8daf-662cc61fa54f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:08.764402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1283/2538] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:46:08.765439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b948bb-3b43-45ef-8daf-662cc61fa54f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:08.776274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b948bb-3b43-45ef-8daf-662cc61fa54f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:08.781407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682c5ba-ec28-4eac-8b1e-6cbcbc8e435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:08.782209 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c5ba-ec28-4eac-8b1e-6cbcbc8e435c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.782537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c5ba-ec28-4eac-8b1e-6cbcbc8e435c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.782775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c5ba-ec28-4eac-8b1e-6cbcbc8e435c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.783424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682c5ba-ec28-4eac-8b1e-6cbcbc8e435c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:08.785217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682c5ba-ec28-4eac-8b1e-6cbcbc8e435c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:08.797620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1284/2539] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:46:08.801781 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682c5ba-ec28-4eac-8b1e-6cbcbc8e435c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:08.812516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682c5ba-ec28-4eac-8b1e-6cbcbc8e435c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:08.817581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f6960-13c7-409d-b91c-0b28de7f923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:08.818600 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f6960-13c7-409d-b91c-0b28de7f923f admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:08.818734 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f6960-13c7-409d-b91c-0b28de7f923f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:08.818934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f6960-13c7-409d-b91c-0b28de7f923f admin admin] PATH_INFO: `/v3/projects/ca298d9092ad4a9489f1624d62b2ac1d/users/33835c5512df4c80a9b7cb9712a5904f/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:08.837436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f6960-13c7-409d-b91c-0b28de7f923f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca298d9092ad4a9489f1624d62b2ac1d, user_id=33835c5512df4c80a9b7cb9712a5904f, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:08.840358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f6960-13c7-409d-b91c-0b28de7f923f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:08.852535 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5898c58b80dd4a0db6d16f31edd1348f', 'request_id': 'req-318f025e-f072-4cad-816b-d619a0eacde6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:08.857880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5898c58b80dd4a0db6d16f31edd1348f', 'request_id': 'req-318f025e-f072-4cad-816b-d619a0eacde6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:08.864243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5898c58b80dd4a0db6d16f31edd1348f', 'request_id': 'req-318f025e-f072-4cad-816b-d619a0eacde6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:08.994202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:08.999623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] The token cache is being invalidate because project 5898c58b80dd4a0db6d16f31edd1348f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:09.004280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:09.004280 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-318f025e-f072-4cad-816b-d619a0eacde6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:09.004470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1285/2540] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:08 2022] PUT /identity/v3/projects/ca298d9092ad4a9489f1624d62b2ac1d/users/33835c5512df4c80a9b7cb9712a5904f/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:09.004674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1256/2541] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:08 2022] DELETE /identity/v3/projects/5898c58b80dd4a0db6d16f31edd1348f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:09.009157 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d4a2-1134-4f56-9d94-e381ab02280c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.009448 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d4a2-1134-4f56-9d94-e381ab02280c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.009721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d4a2-1134-4f56-9d94-e381ab02280c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.014297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce68fe-28ba-48f0-abe7-81631ef4607c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.014675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce68fe-28ba-48f0-abe7-81631ef4607c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.014964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce68fe-28ba-48f0-abe7-81631ef4607c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.035152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-73ce68fe-28ba-48f0-abe7-81631ef4607c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:09.035876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-fc58d4a2-1134-4f56-9d94-e381ab02280c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:09.036414 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-73ce68fe-28ba-48f0-abe7-81631ef4607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:09.037908 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-fc58d4a2-1134-4f56-9d94-e381ab02280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:09.099184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1286/2542] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:09.100571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1257/2543] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:09.108340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc6a9-aac5-4a02-bf8b-94402d2a2d10 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.108727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc6a9-aac5-4a02-bf8b-94402d2a2d10 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.109001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bc6a9-aac5-4a02-bf8b-94402d2a2d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.109688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce68fe-28ba-48f0-abe7-81631ef4607c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:09.111573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-73ce68fe-28ba-48f0-abe7-81631ef4607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:09.128922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e79bc6a9-aac5-4a02-bf8b-94402d2a2d10 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:09.130135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e79bc6a9-aac5-4a02-bf8b-94402d2a2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:09.138989 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1287/2544] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:09.143161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc6a9-aac5-4a02-bf8b-94402d2a2d10 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:09.154744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bc6a9-aac5-4a02-bf8b-94402d2a2d10 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:09.161439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:09.163010 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.163486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.163811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.164703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:46:09.165011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:09.166716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:09.172467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce68fe-28ba-48f0-abe7-81631ef4607c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:09.176504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c88aa8-dbd7-4fe1-a951-91cb8ce7bfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:09.178797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c88aa8-dbd7-4fe1-a951-91cb8ce7bfbc service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.178797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c88aa8-dbd7-4fe1-a951-91cb8ce7bfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.178797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c88aa8-dbd7-4fe1-a951-91cb8ce7bfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.180565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1288/2545] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:46:09.184882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:09.188107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c88aa8-dbd7-4fe1-a951-91cb8ce7bfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:09.189867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c88aa8-dbd7-4fe1-a951-91cb8ce7bfbc service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:09.194967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bec4f-7444-4df7-93b1-36cce6f9af8a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:09.199887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a8fe8-8d1a-4aed-b58c-1d9c2600fe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:09.200333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1258/2546] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:46:09.200561 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a8fe8-8d1a-4aed-b58c-1d9c2600fe23 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.200805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a8fe8-8d1a-4aed-b58c-1d9c2600fe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.201081 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a8fe8-8d1a-4aed-b58c-1d9c2600fe23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.201711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a8fe8-8d1a-4aed-b58c-1d9c2600fe23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:09.203742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a8fe8-8d1a-4aed-b58c-1d9c2600fe23 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:09.320674 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1289/2547] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:09 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:09.325109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c88aa8-dbd7-4fe1-a951-91cb8ce7bfbc service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:09.335877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c88aa8-dbd7-4fe1-a951-91cb8ce7bfbc service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:09.397846 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ce62c-f6cb-453d-a1d4-6a9c4d01ab66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:09.398474 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ce62c-f6cb-453d-a1d4-6a9c4d01ab66 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.398761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ce62c-f6cb-453d-a1d4-6a9c4d01ab66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.399048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ce62c-f6cb-453d-a1d4-6a9c4d01ab66 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.399765 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ce62c-f6cb-453d-a1d4-6a9c4d01ab66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:09.401675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ce62c-f6cb-453d-a1d4-6a9c4d01ab66 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:09.468488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:09.508937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1259/2548] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:09 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:09.513184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a8fe8-8d1a-4aed-b58c-1d9c2600fe23 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:09.525282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a8fe8-8d1a-4aed-b58c-1d9c2600fe23 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:09.530239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4171084-7fe9-4d40-af01-c4fd15d5a12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:09.530851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4171084-7fe9-4d40-af01-c4fd15d5a12f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.531127 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4171084-7fe9-4d40-af01-c4fd15d5a12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.531400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4171084-7fe9-4d40-af01-c4fd15d5a12f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.532046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4171084-7fe9-4d40-af01-c4fd15d5a12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:09.533764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4171084-7fe9-4d40-af01-c4fd15d5a12f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:09.554712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1290/2549] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:09.558495 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ce62c-f6cb-453d-a1d4-6a9c4d01ab66 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:09.570023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ce62c-f6cb-453d-a1d4-6a9c4d01ab66 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:09.575388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fc5cf-35d8-4adc-aed1-43c1d21e93c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:09.576019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc5cf-35d8-4adc-aed1-43c1d21e93c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.576420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc5cf-35d8-4adc-aed1-43c1d21e93c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.576799 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fc5cf-35d8-4adc-aed1-43c1d21e93c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.577569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fc5cf-35d8-4adc-aed1-43c1d21e93c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:09.579329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fc5cf-35d8-4adc-aed1-43c1d21e93c3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:09.594451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1260/2550] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:09.601295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4171084-7fe9-4d40-af01-c4fd15d5a12f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:09.606820 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4171084-7fe9-4d40-af01-c4fd15d5a12f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:09.612023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99391c8e-2b2b-43ab-b648-c8f66a0ba48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:09.612733 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99391c8e-2b2b-43ab-b648-c8f66a0ba48e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.613056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99391c8e-2b2b-43ab-b648-c8f66a0ba48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.613335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99391c8e-2b2b-43ab-b648-c8f66a0ba48e admin admin] PATH_INFO: `/v3/projects/bb4c2d3158f34a39a98c12b2331f80ce/users/ebc0759e092c427f9154da786b3d1ed1/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.629949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99391c8e-2b2b-43ab-b648-c8f66a0ba48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4c2d3158f34a39a98c12b2331f80ce, user_id=ebc0759e092c427f9154da786b3d1ed1, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:09.633768 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99391c8e-2b2b-43ab-b648-c8f66a0ba48e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:09.823498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1291/2551] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:09 2022] PUT /identity/v3/projects/bb4c2d3158f34a39a98c12b2331f80ce/users/ebc0759e092c427f9154da786b3d1ed1/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:09.835670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5cecc-b3a5-4d06-bdaf-a281243ee722 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:09.835967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5cecc-b3a5-4d06-bdaf-a281243ee722 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:09.836420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5cecc-b3a5-4d06-bdaf-a281243ee722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:09.867197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4dd5cecc-b3a5-4d06-bdaf-a281243ee722 None None] MFA Rules not processed for user `ebc0759e092c427f9154da786b3d1ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:09.868552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4dd5cecc-b3a5-4d06-bdaf-a281243ee722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:09.938027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1261/2552] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:09.945167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99391c8e-2b2b-43ab-b648-c8f66a0ba48e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:10.009667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99391c8e-2b2b-43ab-b648-c8f66a0ba48e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:10.014460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfbc75-518d-4ea4-9d8d-aabec1ee5790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:10.015091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfbc75-518d-4ea4-9d8d-aabec1ee5790 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:10.015372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfbc75-518d-4ea4-9d8d-aabec1ee5790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:10.015644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfbc75-518d-4ea4-9d8d-aabec1ee5790 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:10.016301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfbc75-518d-4ea4-9d8d-aabec1ee5790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:10.018187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfbc75-518d-4ea4-9d8d-aabec1ee5790 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:10.164833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1292/2553] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:09 2022] POST /identity/v3/projects => generated 381 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:10.169033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5cecc-b3a5-4d06-bdaf-a281243ee722 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:10.180735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5cecc-b3a5-4d06-bdaf-a281243ee722 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:10.244433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:10.257555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98506c-a7f5-4286-a360-70600b9c6249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:10.258165 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98506c-a7f5-4286-a360-70600b9c6249 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:10.258445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98506c-a7f5-4286-a360-70600b9c6249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:10.258716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98506c-a7f5-4286-a360-70600b9c6249 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:10.259351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98506c-a7f5-4286-a360-70600b9c6249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:10.261367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98506c-a7f5-4286-a360-70600b9c6249 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:10.437755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1262/2554] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:10 2022] POST /identity/v3/users => generated 297 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:10.438087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfbc75-518d-4ea4-9d8d-aabec1ee5790 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:10.449389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfbc75-518d-4ea4-9d8d-aabec1ee5790 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:10.454118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c7ea0-f933-409d-9ec5-0db0b1a875d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:10.454716 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c7ea0-f933-409d-9ec5-0db0b1a875d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:10.455064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c7ea0-f933-409d-9ec5-0db0b1a875d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:10.455322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c7ea0-f933-409d-9ec5-0db0b1a875d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:10.455974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c7ea0-f933-409d-9ec5-0db0b1a875d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:10.457769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c7ea0-f933-409d-9ec5-0db0b1a875d0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:10.469358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1293/2555] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:10.473899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98506c-a7f5-4286-a360-70600b9c6249 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:10.489566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98506c-a7f5-4286-a360-70600b9c6249 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:10.495020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd542c-e8c7-4f9b-a91e-c126df5496de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:10.495596 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd542c-e8c7-4f9b-a91e-c126df5496de admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:10.495955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd542c-e8c7-4f9b-a91e-c126df5496de admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:10.496164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd542c-e8c7-4f9b-a91e-c126df5496de admin admin] PATH_INFO: `/v3/projects/b4efc988f9144d1386c596d3346cbeb0/users/46a9b550d0e14cbe87a57fc8685dd8df/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:10.511878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd542c-e8c7-4f9b-a91e-c126df5496de admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4efc988f9144d1386c596d3346cbeb0, user_id=46a9b550d0e14cbe87a57fc8685dd8df, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:10.514591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd542c-e8c7-4f9b-a91e-c126df5496de admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:10.656270 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1263/2556] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:10 2022] PUT /identity/v3/projects/b4efc988f9144d1386c596d3346cbeb0/users/46a9b550d0e14cbe87a57fc8685dd8df/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:10.660234 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c7ea0-f933-409d-9ec5-0db0b1a875d0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:10.670948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c7ea0-f933-409d-9ec5-0db0b1a875d0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:10.726986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609b151-2a68-40c3-b412-f940f695e09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:10.727498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609b151-2a68-40c3-b412-f940f695e09f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:10.727773 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609b151-2a68-40c3-b412-f940f695e09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:10.728043 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609b151-2a68-40c3-b412-f940f695e09f admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:10.728711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609b151-2a68-40c3-b412-f940f695e09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:10.730356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609b151-2a68-40c3-b412-f940f695e09f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:10.736775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:10.749700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1294/2557] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:10.754064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd542c-e8c7-4f9b-a91e-c126df5496de admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:10.760153 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd542c-e8c7-4f9b-a91e-c126df5496de admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:10.765663 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:10.766358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:10.766642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:10.766977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] PATH_INFO: `/v3/projects/b4efc988f9144d1386c596d3346cbeb0/users/46a9b550d0e14cbe87a57fc8685dd8df/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:10.768974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4efc988f9144d1386c596d3346cbeb0, user_id=46a9b550d0e14cbe87a57fc8685dd8df, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:10.772256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:10.784071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1264/2558] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:10 2022] PUT /identity/v3/projects/b4efc988f9144d1386c596d3346cbeb0/users/46a9b550d0e14cbe87a57fc8685dd8df/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:10.794708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a644c-820a-483c-864d-8db78bb97b0a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:10.795001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a644c-820a-483c-864d-8db78bb97b0a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:10.795282 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a644c-820a-483c-864d-8db78bb97b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:10.825805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-060a644c-820a-483c-864d-8db78bb97b0a None None] MFA Rules not processed for user `46a9b550d0e14cbe87a57fc8685dd8df`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:10.826919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-060a644c-820a-483c-864d-8db78bb97b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:10.913690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1295/2559] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:10.921107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:10.923021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:10.970705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:10.985940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6bd3b-5a30-4f39-ad84-01e746c66bb9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:10.990319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9b8b-f6b6-437b-8486-703eefa6098f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:10.990929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9b8b-f6b6-437b-8486-703eefa6098f service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:10.991210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9b8b-f6b6-437b-8486-703eefa6098f service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:10.991485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9b8b-f6b6-437b-8486-703eefa6098f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:11.001169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff9b8b-f6b6-437b-8486-703eefa6098f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:11.008440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff9b8b-f6b6-437b-8486-703eefa6098f service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:11.020453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1265/2560] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:11.214808 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a644c-820a-483c-864d-8db78bb97b0a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:11.224974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a644c-820a-483c-864d-8db78bb97b0a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:11.228974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363344e1-f899-4d83-8ea1-bf5f427f2c92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:11.229658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363344e1-f899-4d83-8ea1-bf5f427f2c92 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:11.230038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363344e1-f899-4d83-8ea1-bf5f427f2c92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:11.230371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363344e1-f899-4d83-8ea1-bf5f427f2c92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:11.312687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363344e1-f899-4d83-8ea1-bf5f427f2c92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:11.314176 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363344e1-f899-4d83-8ea1-bf5f427f2c92 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:11.329144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1296/2561] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:11.925841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9b8b-f6b6-437b-8486-703eefa6098f service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:12.058289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9b8b-f6b6-437b-8486-703eefa6098f service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:12.063441 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:12.064089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:12.064673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:12.064942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] PATH_INFO: `/v3/users/ebc0759e092c427f9154da786b3d1ed1` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:12.066117 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc0759e092c427f9154da786b3d1ed1)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:12.072075 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:12.085435 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=5,keystone:SELECT=94,keystone:DELETE=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:12.292763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc0759e092c427f9154da786b3d1ed1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:12.293415 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc0759e092c427f9154da786b3d1ed1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:12.302123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc0759e092c427f9154da786b3d1ed1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:12.303040 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc0759e092c427f9154da786b3d1ed1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:12.411292 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc0759e092c427f9154da786b3d1ed1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:12.440400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1266/2562] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:11 2022] DELETE /identity/v3/users/ebc0759e092c427f9154da786b3d1ed1 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:12.445659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f43a9-4933-4a4d-a3d4-9782af74a9e1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:12.446248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f43a9-4933-4a4d-a3d4-9782af74a9e1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:12.446248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f43a9-4933-4a4d-a3d4-9782af74a9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:12.510625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b66f43a9-4933-4a4d-a3d4-9782af74a9e1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:12.513544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b66f43a9-4933-4a4d-a3d4-9782af74a9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:12.530950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:12.585264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1297/2563] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:12.596547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:12.598092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:12.690760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260851db-5c7c-4e4b-979f-1c8433cc6528 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:12.695771 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6544d7-f6fb-46a2-a1d6-0987f01336de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:12.696425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6544d7-f6fb-46a2-a1d6-0987f01336de service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:12.696709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6544d7-f6fb-46a2-a1d6-0987f01336de service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:12.696981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6544d7-f6fb-46a2-a1d6-0987f01336de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:12.707740 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6544d7-f6fb-46a2-a1d6-0987f01336de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:12.709351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6544d7-f6fb-46a2-a1d6-0987f01336de service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:12.721558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1267/2564] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:13.277533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f43a9-4933-4a4d-a3d4-9782af74a9e1 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:13.279078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b66f43a9-4933-4a4d-a3d4-9782af74a9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:13.302657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f43a9-4933-4a4d-a3d4-9782af74a9e1 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:13.307869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:13.309170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:13.309453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:13.309729 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] PATH_INFO: `/v3/projects/bb4c2d3158f34a39a98c12b2331f80ce` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:13.311055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb4c2d3158f34a39a98c12b2331f80ce)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:13.313423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:13.442530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb4c2d3158f34a39a98c12b2331f80ce', 'request_id': 'req-bc1a72df-fa63-4189-8545-94029fc7ca65'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:13.448966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb4c2d3158f34a39a98c12b2331f80ce', 'request_id': 'req-bc1a72df-fa63-4189-8545-94029fc7ca65'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:13.542296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb4c2d3158f34a39a98c12b2331f80ce', 'request_id': 'req-bc1a72df-fa63-4189-8545-94029fc7ca65'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:13.562006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] The token cache is being invalidate because project bb4c2d3158f34a39a98c12b2331f80ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:13.562302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:13.563430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:13.565446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1298/2565] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:13 2022] DELETE /identity/v3/projects/bb4c2d3158f34a39a98c12b2331f80ce => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:13.570629 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6544d7-f6fb-46a2-a1d6-0987f01336de service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:13.589542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6544d7-f6fb-46a2-a1d6-0987f01336de service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:13.677854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:13.677854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:13.678518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:13.678805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] PATH_INFO: `/v3/users/46a9b550d0e14cbe87a57fc8685dd8df` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:13.680411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a9b550d0e14cbe87a57fc8685dd8df)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:13.682384 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:13.709948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:13.929163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a9b550d0e14cbe87a57fc8685dd8df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:13.930465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a9b550d0e14cbe87a57fc8685dd8df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:13.941099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a9b550d0e14cbe87a57fc8685dd8df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:13.942310 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a9b550d0e14cbe87a57fc8685dd8df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:14.090671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-203a4a58-942b-44ba-b936-885d513c3993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a9b550d0e14cbe87a57fc8685dd8df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:14.111624 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1268/2566] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:13 2022] DELETE /identity/v3/users/46a9b550d0e14cbe87a57fc8685dd8df => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:14.852618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:14.854147 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:14.869292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a72df-fa63-4189-8545-94029fc7ca65 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:14.941340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:14.942235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:14.942527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:14.943351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] PATH_INFO: `/v3/projects/b4efc988f9144d1386c596d3346cbeb0` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:14.944681 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4efc988f9144d1386c596d3346cbeb0)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:14.946933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:15.105395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:15.113013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4efc988f9144d1386c596d3346cbeb0', 'request_id': 'req-b20d947b-f168-4315-8be5-1b648721521f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:15.121007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4efc988f9144d1386c596d3346cbeb0', 'request_id': 'req-b20d947b-f168-4315-8be5-1b648721521f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:15.224673 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4efc988f9144d1386c596d3346cbeb0', 'request_id': 'req-b20d947b-f168-4315-8be5-1b648721521f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:15.251417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] The token cache is being invalidate because project b4efc988f9144d1386c596d3346cbeb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:15.251718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:15.252422 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b20d947b-f168-4315-8be5-1b648721521f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:15.267775 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1299/2567] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:14 2022] DELETE /identity/v3/projects/b4efc988f9144d1386c596d3346cbeb0 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:15.272371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae08b14-0d85-4001-849f-f7c2e6d89c7a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.273227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae08b14-0d85-4001-849f-f7c2e6d89c7a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.273656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae08b14-0d85-4001-849f-f7c2e6d89c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.375039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-eae08b14-0d85-4001-849f-f7c2e6d89c7a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:15.376970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-eae08b14-0d85-4001-849f-f7c2e6d89c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:15.459955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1269/2568] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:15.468839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20b74-a6be-40df-bdec-874a08c92a7a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.469498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20b74-a6be-40df-bdec-874a08c92a7a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.469947 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e20b74-a6be-40df-bdec-874a08c92a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.498769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-73e20b74-a6be-40df-bdec-874a08c92a7a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:15.500501 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-73e20b74-a6be-40df-bdec-874a08c92a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:15.512300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1300/2569] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:15.516160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae08b14-0d85-4001-849f-f7c2e6d89c7a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:15.531365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae08b14-0d85-4001-849f-f7c2e6d89c7a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:15.536924 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:15.537754 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.538150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.538536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.539401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:46:15.539805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:15.541635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:15.558896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1270/2570] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:46:15.560007 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e20b74-a6be-40df-bdec-874a08c92a7a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:15.572161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e20b74-a6be-40df-bdec-874a08c92a7a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:15.579524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feab020-072a-4f45-8606-c1325b898213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:15.580159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feab020-072a-4f45-8606-c1325b898213 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.580532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feab020-072a-4f45-8606-c1325b898213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.580809 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feab020-072a-4f45-8606-c1325b898213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.581473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feab020-072a-4f45-8606-c1325b898213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:15.583842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feab020-072a-4f45-8606-c1325b898213 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:15.684095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1301/2571] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:15 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:15.688573 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:15.709143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e525a-4f14-45df-8771-67e6ea87b664 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:15.714059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3020e-9869-4bc5-8065-41d5a0548074 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.714059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3020e-9869-4bc5-8065-41d5a0548074 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.714059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3020e-9869-4bc5-8065-41d5a0548074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.746016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-59f3020e-9869-4bc5-8065-41d5a0548074 None None] MFA Rules not processed for user `33835c5512df4c80a9b7cb9712a5904f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:15.747890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-59f3020e-9869-4bc5-8065-41d5a0548074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:15.788010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af4da-069a-48f9-bd2a-101969e7112a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:15.788572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af4da-069a-48f9-bd2a-101969e7112a admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.789014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af4da-069a-48f9-bd2a-101969e7112a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.789356 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af4da-069a-48f9-bd2a-101969e7112a admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.790144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06af4da-069a-48f9-bd2a-101969e7112a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:15.793132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06af4da-069a-48f9-bd2a-101969e7112a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:15.803180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:15.827098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1302/2572] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:15.846006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3020e-9869-4bc5-8065-41d5a0548074 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:15.847468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-59f3020e-9869-4bc5-8065-41d5a0548074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:15.904247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1271/2573] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:15 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:15.909907 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af4da-069a-48f9-bd2a-101969e7112a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:15.921066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af4da-069a-48f9-bd2a-101969e7112a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:15.926251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8790d19-6b53-4cff-87a6-5478ff6f5940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:15.926960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790d19-6b53-4cff-87a6-5478ff6f5940 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.928299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790d19-6b53-4cff-87a6-5478ff6f5940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.928299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8790d19-6b53-4cff-87a6-5478ff6f5940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.929134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8790d19-6b53-4cff-87a6-5478ff6f5940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:15.930845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8790d19-6b53-4cff-87a6-5478ff6f5940 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:15.942528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1272/2574] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:15.946877 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8790d19-6b53-4cff-87a6-5478ff6f5940 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:15.957513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8790d19-6b53-4cff-87a6-5478ff6f5940 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:15.962801 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c44c6f-d0d3-44d5-851c-40af4ea207bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:15.964136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44c6f-d0d3-44d5-851c-40af4ea207bb admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.965274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44c6f-d0d3-44d5-851c-40af4ea207bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.965274 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44c6f-d0d3-44d5-851c-40af4ea207bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.965602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c44c6f-d0d3-44d5-851c-40af4ea207bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:15.967389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c44c6f-d0d3-44d5-851c-40af4ea207bb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:15.967856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:15.970576 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3020e-9869-4bc5-8065-41d5a0548074 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:15.977589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2bf0bd-c4f9-4347-a12a-fcbb03fee902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:15.978288 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bf0bd-c4f9-4347-a12a-fcbb03fee902 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:15.978646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bf0bd-c4f9-4347-a12a-fcbb03fee902 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:15.978984 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2bf0bd-c4f9-4347-a12a-fcbb03fee902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:15.984078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1273/2575] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:46:15.988451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c44c6f-d0d3-44d5-851c-40af4ea207bb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:15.988664 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bf0bd-c4f9-4347-a12a-fcbb03fee902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:15.989715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2bf0bd-c4f9-4347-a12a-fcbb03fee902 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:15.998508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c44c6f-d0d3-44d5-851c-40af4ea207bb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:16.000606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1303/2576] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:46:16.003666 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:16.005939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:16.005939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:16.005939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] PATH_INFO: `/v3/projects/d1849d4634f74d3eae13ce4c5cfd03d5/users/9a171f980e324989b187a273e250625a/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:16.023631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1849d4634f74d3eae13ce4c5cfd03d5, user_id=9a171f980e324989b187a273e250625a, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:16.025880 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:16.068871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1274/2577] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:15 2022] PUT /identity/v3/projects/d1849d4634f74d3eae13ce4c5cfd03d5/users/9a171f980e324989b187a273e250625a/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:16.135866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75454d8b-408a-48ce-b77c-847148151614 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:16.136160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75454d8b-408a-48ce-b77c-847148151614 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:16.136481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75454d8b-408a-48ce-b77c-847148151614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:16.165831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-75454d8b-408a-48ce-b77c-847148151614 None None] MFA Rules not processed for user `9a171f980e324989b187a273e250625a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:16.167067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-75454d8b-408a-48ce-b77c-847148151614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:16.252238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1304/2578] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:16.261293 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:16.263422 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:16.325889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92978a63-619b-4482-8b71-6a184c4121c9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:16.329688 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43ecf8-b1eb-432d-8c09-ca4737e9213f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:16.330501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ecf8-b1eb-432d-8c09-ca4737e9213f service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:16.331578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ecf8-b1eb-432d-8c09-ca4737e9213f service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:16.331892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43ecf8-b1eb-432d-8c09-ca4737e9213f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:16.341860 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43ecf8-b1eb-432d-8c09-ca4737e9213f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:16.343595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43ecf8-b1eb-432d-8c09-ca4737e9213f service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:16.353601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1275/2579] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:16.373693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75454d8b-408a-48ce-b77c-847148151614 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:16.464271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75454d8b-408a-48ce-b77c-847148151614 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:16.468855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:16.469482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:16.469766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:16.470042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] PATH_INFO: `/v3/users/9a171f980e324989b187a273e250625a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:16.471277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a171f980e324989b187a273e250625a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:16.473199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:16.669042 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a171f980e324989b187a273e250625a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:16.670044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a171f980e324989b187a273e250625a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:16.677862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a171f980e324989b187a273e250625a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:16.770204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a171f980e324989b187a273e250625a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:16.771510 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a171f980e324989b187a273e250625a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:16.788324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:16.793497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1305/2580] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:16 2022] DELETE /identity/v3/users/9a171f980e324989b187a273e250625a => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:16.798192 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78283-0115-4edc-85e0-02add09e722c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:16.798749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78283-0115-4edc-85e0-02add09e722c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:16.799034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd78283-0115-4edc-85e0-02add09e722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:16.821345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-dfd78283-0115-4edc-85e0-02add09e722c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:16.822576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dfd78283-0115-4edc-85e0-02add09e722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:16.890355 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1276/2581] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:16.901511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:16.902974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:16.975487 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994733f-4b21-4914-8019-a008f7414329 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:16.980203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d06334-f478-48eb-bb4b-8ad4fc53f7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:16.981377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06334-f478-48eb-bb4b-8ad4fc53f7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:16.981377 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06334-f478-48eb-bb4b-8ad4fc53f7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:16.981650 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d06334-f478-48eb-bb4b-8ad4fc53f7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:16.992608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d06334-f478-48eb-bb4b-8ad4fc53f7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:16.994177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d06334-f478-48eb-bb4b-8ad4fc53f7c4 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:17.008116 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1306/2582] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:17.484462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd78283-0115-4edc-85e0-02add09e722c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:17.490005 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-dfd78283-0115-4edc-85e0-02add09e722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:17.522329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd78283-0115-4edc-85e0-02add09e722c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:17.522329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:17.522329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:17.522329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:17.523160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] PATH_INFO: `/v3/projects/d1849d4634f74d3eae13ce4c5cfd03d5` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:17.523160 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1849d4634f74d3eae13ce4c5cfd03d5)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:17.525506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:17.664464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=3,keystone:SELECT=94,keystone:DELETE=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:17.668105 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1849d4634f74d3eae13ce4c5cfd03d5', 'request_id': 'req-10104f6b-71c0-4e75-b724-5270d1921895'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:17.673800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1849d4634f74d3eae13ce4c5cfd03d5', 'request_id': 'req-10104f6b-71c0-4e75-b724-5270d1921895'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:17.680385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1849d4634f74d3eae13ce4c5cfd03d5', 'request_id': 'req-10104f6b-71c0-4e75-b724-5270d1921895'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:17.817477 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] The token cache is being invalidate because project d1849d4634f74d3eae13ce4c5cfd03d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:17.818946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:17.821221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-10104f6b-71c0-4e75-b724-5270d1921895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:17.823241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1277/2583] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:17 2022] DELETE /identity/v3/projects/d1849d4634f74d3eae13ce4c5cfd03d5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:17.828258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22b92e-236a-44ff-af18-43009938e221 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:17.828551 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22b92e-236a-44ff-af18-43009938e221 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:17.828825 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22b92e-236a-44ff-af18-43009938e221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:17.850903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-3e22b92e-236a-44ff-af18-43009938e221 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:17.854113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-3e22b92e-236a-44ff-af18-43009938e221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:17.931854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1307/2584] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:17.939805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72c3c8-1f87-4bb0-ab7a-a2ab7aea10a2 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:17.940126 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72c3c8-1f87-4bb0-ab7a-a2ab7aea10a2 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:17.940462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72c3c8-1f87-4bb0-ab7a-a2ab7aea10a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:17.961431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ee72c3c8-1f87-4bb0-ab7a-a2ab7aea10a2 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:17.962684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ee72c3c8-1f87-4bb0-ab7a-a2ab7aea10a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:17.971257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1278/2585] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:17.976245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22b92e-236a-44ff-af18-43009938e221 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:17.986054 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22b92e-236a-44ff-af18-43009938e221 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:17.990701 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:17.991302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:17.991666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:17.991965 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:17.992660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:46:17.992968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:17.994568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:18.006890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1308/2586] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:46:18.011323 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72c3c8-1f87-4bb0-ab7a-a2ab7aea10a2 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:18.021685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72c3c8-1f87-4bb0-ab7a-a2ab7aea10a2 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:18.028362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ec7b5-76cb-4039-95ff-ba83d0bd9629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:18.028981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ec7b5-76cb-4039-95ff-ba83d0bd9629 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:18.029398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ec7b5-76cb-4039-95ff-ba83d0bd9629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:18.029678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ec7b5-76cb-4039-95ff-ba83d0bd9629 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:18.030698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ec7b5-76cb-4039-95ff-ba83d0bd9629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:18.033584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ec7b5-76cb-4039-95ff-ba83d0bd9629 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:18.156694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1279/2587] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:18 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:18.160927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:18.172352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74338824-ba38-410c-b9fe-4b49670dd2e7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:18.256555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd73b4-b0a1-4902-bf89-a2e50d59c160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:18.257229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd73b4-b0a1-4902-bf89-a2e50d59c160 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:18.257451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd73b4-b0a1-4902-bf89-a2e50d59c160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:18.257724 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd73b4-b0a1-4902-bf89-a2e50d59c160 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:18.258402 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd73b4-b0a1-4902-bf89-a2e50d59c160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:18.260450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd73b4-b0a1-4902-bf89-a2e50d59c160 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:18.384489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1309/2588] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:18 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:18.388721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ec7b5-76cb-4039-95ff-ba83d0bd9629 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:18.405597 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ec7b5-76cb-4039-95ff-ba83d0bd9629 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:18.410743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e63bb-e36f-43a9-b591-1a48e12d56a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:18.412028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e63bb-e36f-43a9-b591-1a48e12d56a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:18.412555 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e63bb-e36f-43a9-b591-1a48e12d56a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:18.413145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e63bb-e36f-43a9-b591-1a48e12d56a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:18.413887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e63bb-e36f-43a9-b591-1a48e12d56a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:18.415631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e63bb-e36f-43a9-b591-1a48e12d56a2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:18.430221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1280/2589] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:18.434315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd73b4-b0a1-4902-bf89-a2e50d59c160 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:18.447347 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd73b4-b0a1-4902-bf89-a2e50d59c160 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:18.452341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924e8bb-d47c-46ab-a06e-64e299d3e2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:18.452964 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924e8bb-d47c-46ab-a06e-64e299d3e2fe admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:18.454791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924e8bb-d47c-46ab-a06e-64e299d3e2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:18.454791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924e8bb-d47c-46ab-a06e-64e299d3e2fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:18.454791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924e8bb-d47c-46ab-a06e-64e299d3e2fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:18.455809 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924e8bb-d47c-46ab-a06e-64e299d3e2fe admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:18.471107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1310/2590] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:18.472220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e63bb-e36f-43a9-b591-1a48e12d56a2 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:18.484387 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e63bb-e36f-43a9-b591-1a48e12d56a2 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:18.489330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12232596-84bf-434e-9dab-6b3f18a3fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:18.490129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12232596-84bf-434e-9dab-6b3f18a3fa13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:18.490410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12232596-84bf-434e-9dab-6b3f18a3fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:18.490895 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12232596-84bf-434e-9dab-6b3f18a3fa13 admin admin] PATH_INFO: `/v3/projects/2623fa2f0d6e49eface45be1d30dcff9/users/cc402f861aca4d53872bdd8922b4ee24/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:18.498314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ae5b3-1577-42c9-bff9-7c204e10abc1 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:18.498461 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ae5b3-1577-42c9-bff9-7c204e10abc1 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:18.498699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ae5b3-1577-42c9-bff9-7c204e10abc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:18.509107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12232596-84bf-434e-9dab-6b3f18a3fa13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2623fa2f0d6e49eface45be1d30dcff9, user_id=cc402f861aca4d53872bdd8922b4ee24, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:18.512125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12232596-84bf-434e-9dab-6b3f18a3fa13 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:18.529532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d80ae5b3-1577-42c9-bff9-7c204e10abc1 None None] MFA Rules not processed for user `33835c5512df4c80a9b7cb9712a5904f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:18.529532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d80ae5b3-1577-42c9-bff9-7c204e10abc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:18.544955 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:18.616839 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1311/2591] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:18.693271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1281/2592] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:18 2022] PUT /identity/v3/projects/2623fa2f0d6e49eface45be1d30dcff9/users/cc402f861aca4d53872bdd8922b4ee24/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:18.705240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2b58-6ba4-4fa3-901c-4c7280fbd7df None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:18.705838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2b58-6ba4-4fa3-901c-4c7280fbd7df None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:18.706310 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad2b58-6ba4-4fa3-901c-4c7280fbd7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:18.737386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0aad2b58-6ba4-4fa3-901c-4c7280fbd7df None None] MFA Rules not processed for user `cc402f861aca4d53872bdd8922b4ee24`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:18.738850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0aad2b58-6ba4-4fa3-901c-4c7280fbd7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:18.826862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1312/2593] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:18.831447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12232596-84bf-434e-9dab-6b3f18a3fa13 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:18.907777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12232596-84bf-434e-9dab-6b3f18a3fa13 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:18.913503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa147b-d6fe-4bdf-94ef-abb50c19585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:18.914245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa147b-d6fe-4bdf-94ef-abb50c19585f admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:18.914532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa147b-d6fe-4bdf-94ef-abb50c19585f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:18.914841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa147b-d6fe-4bdf-94ef-abb50c19585f admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:18.917828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa147b-d6fe-4bdf-94ef-abb50c19585f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:18.929643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa147b-d6fe-4bdf-94ef-abb50c19585f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:19.061305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1282/2594] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:18 2022] POST /identity/v3/projects => generated 375 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:19.065948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad2b58-6ba4-4fa3-901c-4c7280fbd7df None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:19.084965 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad2b58-6ba4-4fa3-901c-4c7280fbd7df None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:19.161623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2afa3-0e84-4f57-9d83-47fb9ba8ad38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:19.162546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2afa3-0e84-4f57-9d83-47fb9ba8ad38 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:19.163023 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2afa3-0e84-4f57-9d83-47fb9ba8ad38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:19.163468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2afa3-0e84-4f57-9d83-47fb9ba8ad38 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:19.164307 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2afa3-0e84-4f57-9d83-47fb9ba8ad38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:19.168492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2afa3-0e84-4f57-9d83-47fb9ba8ad38 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:19.251057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1313/2595] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:19 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:19.255854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa147b-d6fe-4bdf-94ef-abb50c19585f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:19.271686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa147b-d6fe-4bdf-94ef-abb50c19585f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:19.294307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d857f-5989-4856-96fc-d3bacc494ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:19.294974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d857f-5989-4856-96fc-d3bacc494ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:19.295257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d857f-5989-4856-96fc-d3bacc494ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:19.295531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d857f-5989-4856-96fc-d3bacc494ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:19.296277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d857f-5989-4856-96fc-d3bacc494ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:19.298085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d857f-5989-4856-96fc-d3bacc494ce0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:19.315889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1283/2596] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:19.317372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2afa3-0e84-4f57-9d83-47fb9ba8ad38 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:19.339995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2afa3-0e84-4f57-9d83-47fb9ba8ad38 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:19.347035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79238a9-6f09-4355-b610-5caf3ea7a7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:19.348237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79238a9-6f09-4355-b610-5caf3ea7a7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:19.348717 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79238a9-6f09-4355-b610-5caf3ea7a7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:19.349854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79238a9-6f09-4355-b610-5caf3ea7a7f1 admin admin] PATH_INFO: `/v3/projects/e8ea0cb54d01497598e653993da58e23/users/8bde89e4469b4cbdaa03194874020828/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:19.377252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79238a9-6f09-4355-b610-5caf3ea7a7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8ea0cb54d01497598e653993da58e23, user_id=8bde89e4469b4cbdaa03194874020828, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:19.381426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79238a9-6f09-4355-b610-5caf3ea7a7f1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:19.440928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1314/2597] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:19 2022] PUT /identity/v3/projects/e8ea0cb54d01497598e653993da58e23/users/8bde89e4469b4cbdaa03194874020828/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:19.445115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d857f-5989-4856-96fc-d3bacc494ce0 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:19.457792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d857f-5989-4856-96fc-d3bacc494ce0 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:19.507399 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:19.529939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf654bc-5e18-4fc8-8be9-9a676c96c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:19.530698 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf654bc-5e18-4fc8-8be9-9a676c96c4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:19.530968 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf654bc-5e18-4fc8-8be9-9a676c96c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:19.531247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf654bc-5e18-4fc8-8be9-9a676c96c4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:19.532062 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf654bc-5e18-4fc8-8be9-9a676c96c4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:19.534605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf654bc-5e18-4fc8-8be9-9a676c96c4a7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:19.551246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1284/2598] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:19.555315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79238a9-6f09-4355-b610-5caf3ea7a7f1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:19.572927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79238a9-6f09-4355-b610-5caf3ea7a7f1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:19.578454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:19.579214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:19.580215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:19.580215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] PATH_INFO: `/v3/projects/e8ea0cb54d01497598e653993da58e23/users/8bde89e4469b4cbdaa03194874020828/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:19.582231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8ea0cb54d01497598e653993da58e23, user_id=8bde89e4469b4cbdaa03194874020828, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:19.587598 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:19.591766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:19.607480 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1315/2599] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:19 2022] PUT /identity/v3/projects/e8ea0cb54d01497598e653993da58e23/users/8bde89e4469b4cbdaa03194874020828/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:19.619016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698a58e-12b9-49cf-8993-7cbc4a170814 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:19.619397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698a58e-12b9-49cf-8993-7cbc4a170814 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:19.619763 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698a58e-12b9-49cf-8993-7cbc4a170814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:19.659067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-9698a58e-12b9-49cf-8993-7cbc4a170814 None None] MFA Rules not processed for user `8bde89e4469b4cbdaa03194874020828`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:19.660419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9698a58e-12b9-49cf-8993-7cbc4a170814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:19.751246 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1285/2600] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:19.769057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:19.772455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:19.869498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684dd00-421f-4d36-b57b-2d54a3b79cbf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:19.873457 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a3b6b-b19e-4ca0-8ac1-3de8d34f5423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:19.874167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a3b6b-b19e-4ca0-8ac1-3de8d34f5423 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:19.874538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a3b6b-b19e-4ca0-8ac1-3de8d34f5423 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:19.874887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a3b6b-b19e-4ca0-8ac1-3de8d34f5423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:19.951358 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a3b6b-b19e-4ca0-8ac1-3de8d34f5423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:19.953319 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a3b6b-b19e-4ca0-8ac1-3de8d34f5423 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:19.965291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1316/2601] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:23.271158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698a58e-12b9-49cf-8993-7cbc4a170814 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:23.334690 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698a58e-12b9-49cf-8993-7cbc4a170814 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:23.339334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:23.340010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:23.340404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:23.340685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] PATH_INFO: `/v3/users/cc402f861aca4d53872bdd8922b4ee24` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:23.341823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc402f861aca4d53872bdd8922b4ee24)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:23.343776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:23.625429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc402f861aca4d53872bdd8922b4ee24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:23.626365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc402f861aca4d53872bdd8922b4ee24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:23.640789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc402f861aca4d53872bdd8922b4ee24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:23.641536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc402f861aca4d53872bdd8922b4ee24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:23.799358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc402f861aca4d53872bdd8922b4ee24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:23.801783 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:23.822010 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1286/2602] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:23 2022] DELETE /identity/v3/users/cc402f861aca4d53872bdd8922b4ee24 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:23.828047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a923e90-09ec-4908-8e2c-3b61c314c804 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:23.828612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a923e90-09ec-4908-8e2c-3b61c314c804 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:23.828776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a923e90-09ec-4908-8e2c-3b61c314c804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:23.851009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-1a923e90-09ec-4908-8e2c-3b61c314c804 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:23.852309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1a923e90-09ec-4908-8e2c-3b61c314c804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:23.916855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1317/2603] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:23.928828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:23.930290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:23.989419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f8fe2-d814-4988-8b4e-e14f78252f1a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:23.993730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f85774-1a67-4edf-9dc8-0c810b768fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:23.994407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f85774-1a67-4edf-9dc8-0c810b768fcb service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:23.994622 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f85774-1a67-4edf-9dc8-0c810b768fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:23.994894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f85774-1a67-4edf-9dc8-0c810b768fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:24.005238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f85774-1a67-4edf-9dc8-0c810b768fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:24.007080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f85774-1a67-4edf-9dc8-0c810b768fcb service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:24.018017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1287/2604] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:24.804108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a923e90-09ec-4908-8e2c-3b61c314c804 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:24.805742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1a923e90-09ec-4908-8e2c-3b61c314c804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:24.824679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a923e90-09ec-4908-8e2c-3b61c314c804 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:24.831683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:24.833475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:24.834090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:24.834696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] PATH_INFO: `/v3/projects/2623fa2f0d6e49eface45be1d30dcff9` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:24.837190 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2623fa2f0d6e49eface45be1d30dcff9)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:24.841349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:25.030715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:25.035660 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2623fa2f0d6e49eface45be1d30dcff9', 'request_id': 'req-bc00152a-f462-4caf-96bf-3d50dd91ad86'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.045874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2623fa2f0d6e49eface45be1d30dcff9', 'request_id': 'req-bc00152a-f462-4caf-96bf-3d50dd91ad86'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.222747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2623fa2f0d6e49eface45be1d30dcff9', 'request_id': 'req-bc00152a-f462-4caf-96bf-3d50dd91ad86'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.236290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] The token cache is being invalidate because project 2623fa2f0d6e49eface45be1d30dcff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:25.236692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.237434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.239400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1318/2605] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:24 2022] DELETE /identity/v3/projects/2623fa2f0d6e49eface45be1d30dcff9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:25.246808 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f85774-1a67-4edf-9dc8-0c810b768fcb service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:25.257407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f85774-1a67-4edf-9dc8-0c810b768fcb service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:25.331373 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:25.332023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:25.332345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:25.332628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] PATH_INFO: `/v3/users/8bde89e4469b4cbdaa03194874020828` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:25.335260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bde89e4469b4cbdaa03194874020828)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:25.337976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:25.581975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bde89e4469b4cbdaa03194874020828'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.583750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bde89e4469b4cbdaa03194874020828'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.599718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bde89e4469b4cbdaa03194874020828'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.601346 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bde89e4469b4cbdaa03194874020828'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.718569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7d165456-4f98-49b9-92af-300ad276b6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bde89e4469b4cbdaa03194874020828'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:25.755937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1288/2606] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:25 2022] DELETE /identity/v3/users/8bde89e4469b4cbdaa03194874020828 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:26.481179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:26.482747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:26.500860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00152a-f462-4caf-96bf-3d50dd91ad86 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:26.573707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:26.574408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:26.574685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:26.574957 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] PATH_INFO: `/v3/projects/e8ea0cb54d01497598e653993da58e23` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:26.576130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8ea0cb54d01497598e653993da58e23)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:26.578129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:26.710787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8ea0cb54d01497598e653993da58e23', 'request_id': 'req-e548ed1d-94b1-4edf-84e1-914bf30e1776'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:26.710787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8ea0cb54d01497598e653993da58e23', 'request_id': 'req-e548ed1d-94b1-4edf-84e1-914bf30e1776'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:26.828107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8ea0cb54d01497598e653993da58e23', 'request_id': 'req-e548ed1d-94b1-4edf-84e1-914bf30e1776'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:26.842732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] The token cache is being invalidate because project e8ea0cb54d01497598e653993da58e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:26.843027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:26.843708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e548ed1d-94b1-4edf-84e1-914bf30e1776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:26.845274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1319/2607] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:26 2022] DELETE /identity/v3/projects/e8ea0cb54d01497598e653993da58e23 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:26.851434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b6b1e-79b3-4012-b2bc-b30f821d4e78 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:26.852028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b6b1e-79b3-4012-b2bc-b30f821d4e78 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:26.852028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b6b1e-79b3-4012-b2bc-b30f821d4e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:26.861171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:26.873417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3b6b6b1e-79b3-4012-b2bc-b30f821d4e78 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:26.875200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3b6b6b1e-79b3-4012-b2bc-b30f821d4e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:26.951260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1289/2608] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:26.959751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef01c84-b04b-490e-8018-dd975ed8a849 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:26.960031 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef01c84-b04b-490e-8018-dd975ed8a849 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:26.960351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef01c84-b04b-490e-8018-dd975ed8a849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:26.981682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-0ef01c84-b04b-490e-8018-dd975ed8a849 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:26.982907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0ef01c84-b04b-490e-8018-dd975ed8a849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:26.991342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1320/2609] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:26.995660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b6b1e-79b3-4012-b2bc-b30f821d4e78 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:27.007437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b6b1e-79b3-4012-b2bc-b30f821d4e78 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:27.013210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:27.013913 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:27.014428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:27.014738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:27.015473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:46:27.015855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:27.017909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:27.031458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1290/2610] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:46:27.035835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef01c84-b04b-490e-8018-dd975ed8a849 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:27.052619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef01c84-b04b-490e-8018-dd975ed8a849 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:27.057383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53f57a-78a1-44c3-acf5-eabf40456979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:27.058024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53f57a-78a1-44c3-acf5-eabf40456979 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:27.058304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53f57a-78a1-44c3-acf5-eabf40456979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:27.058578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53f57a-78a1-44c3-acf5-eabf40456979 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:27.059253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53f57a-78a1-44c3-acf5-eabf40456979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:27.061214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53f57a-78a1-44c3-acf5-eabf40456979 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:27.338450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1321/2611] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:27 2022] POST /identity/v3/projects => generated 383 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:27.342813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:27.355216 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547607bb-ce2e-484c-8b45-5819174f4879 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:27.419473 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8914af-c005-438f-a592-3294bf07876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:27.420076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8914af-c005-438f-a592-3294bf07876d admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:27.420420 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8914af-c005-438f-a592-3294bf07876d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:27.420671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8914af-c005-438f-a592-3294bf07876d admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:27.421312 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8914af-c005-438f-a592-3294bf07876d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:27.423254 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8914af-c005-438f-a592-3294bf07876d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:27.500661 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1291/2612] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:27 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:27.505926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53f57a-78a1-44c3-acf5-eabf40456979 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:27.517073 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53f57a-78a1-44c3-acf5-eabf40456979 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:27.522613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3149163-7046-4a3f-8217-6f3acefa6a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:27.523229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3149163-7046-4a3f-8217-6f3acefa6a93 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:27.523504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3149163-7046-4a3f-8217-6f3acefa6a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:27.523774 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3149163-7046-4a3f-8217-6f3acefa6a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:27.524454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3149163-7046-4a3f-8217-6f3acefa6a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:27.526103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3149163-7046-4a3f-8217-6f3acefa6a93 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:27.539341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1322/2613] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:27.544393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8914af-c005-438f-a592-3294bf07876d admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:27.555416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8914af-c005-438f-a592-3294bf07876d admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:27.561076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83e2ff-e341-4e0c-a4fe-a7e2daa4d375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:27.561708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83e2ff-e341-4e0c-a4fe-a7e2daa4d375 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:27.561984 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83e2ff-e341-4e0c-a4fe-a7e2daa4d375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:27.562278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83e2ff-e341-4e0c-a4fe-a7e2daa4d375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:27.563226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83e2ff-e341-4e0c-a4fe-a7e2daa4d375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:27.564990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83e2ff-e341-4e0c-a4fe-a7e2daa4d375 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:27.578210 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1292/2614] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:27.582615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3149163-7046-4a3f-8217-6f3acefa6a93 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:27.603341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3149163-7046-4a3f-8217-6f3acefa6a93 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:27.603341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:27.603341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:27.603341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:27.603924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] PATH_INFO: `/v3/projects/6f61a8f100cf4d39928e4d596741e1bb/users/63c0c061be824eae8d217d7ea32dcd92/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:27.619785 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f61a8f100cf4d39928e4d596741e1bb, user_id=63c0c061be824eae8d217d7ea32dcd92, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:27.622456 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:27.766523 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:27.775565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1323/2615] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:27 2022] PUT /identity/v3/projects/6f61a8f100cf4d39928e4d596741e1bb/users/63c0c061be824eae8d217d7ea32dcd92/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:27.779131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021301c0-7d3d-429e-bcb3-7b24b7622558 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:27.779604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021301c0-7d3d-429e-bcb3-7b24b7622558 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:27.780001 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021301c0-7d3d-429e-bcb3-7b24b7622558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:27.801257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-021301c0-7d3d-429e-bcb3-7b24b7622558 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:27.803283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-021301c0-7d3d-429e-bcb3-7b24b7622558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:27.878171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1293/2616] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:27.889176 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:27.891121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:27.960744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569a3a9-ba63-43f8-9bd8-fb83b3836dae admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:27.968112 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac0fd1-116f-4086-8656-4340a02e9fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:27.983638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac0fd1-116f-4086-8656-4340a02e9fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:27.983638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac0fd1-116f-4086-8656-4340a02e9fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:27.983638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac0fd1-116f-4086-8656-4340a02e9fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:27.983638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac0fd1-116f-4086-8656-4340a02e9fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:27.984201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac0fd1-116f-4086-8656-4340a02e9fe0 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:27.997408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1324/2617] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:34.955389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651564-d846-45c1-83cb-b212cbbea9bd None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:34.955816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651564-d846-45c1-83cb-b212cbbea9bd None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:34.955951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651564-d846-45c1-83cb-b212cbbea9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:34.985684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:34.996828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c8651564-d846-45c1-83cb-b212cbbea9bd None None] MFA Rules not processed for user `63c0c061be824eae8d217d7ea32dcd92`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:34.998096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c8651564-d846-45c1-83cb-b212cbbea9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:35.154015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1294/2618] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:35.155557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac0fd1-116f-4086-8656-4340a02e9fe0 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:35.184252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac0fd1-116f-4086-8656-4340a02e9fe0 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:35.196012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6898-5c99-4307-b4ba-5b8e5aef800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:35.197662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6898-5c99-4307-b4ba-5b8e5aef800a admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:35.198498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6898-5c99-4307-b4ba-5b8e5aef800a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:35.199271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6898-5c99-4307-b4ba-5b8e5aef800a admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:35.200855 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b6898-5c99-4307-b4ba-5b8e5aef800a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:35.208361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b6898-5c99-4307-b4ba-5b8e5aef800a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:35.399059 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1325/2619] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:35 2022] POST /identity/v3/projects => generated 385 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:35.399635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8651564-d846-45c1-83cb-b212cbbea9bd None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:35.418584 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8651564-d846-45c1-83cb-b212cbbea9bd None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:35.500761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbc835-8bb7-472f-a5dc-eedc748e95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:35.501682 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbc835-8bb7-472f-a5dc-eedc748e95f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:35.501937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbc835-8bb7-472f-a5dc-eedc748e95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:35.502339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbc835-8bb7-472f-a5dc-eedc748e95f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:35.504260 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbc835-8bb7-472f-a5dc-eedc748e95f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:35.510800 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbc835-8bb7-472f-a5dc-eedc748e95f7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:35.653543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1295/2620] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:35 2022] POST /identity/v3/users => generated 299 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:35.655754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6898-5c99-4307-b4ba-5b8e5aef800a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:35.675406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6898-5c99-4307-b4ba-5b8e5aef800a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:35.681715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9a2fc-dd64-442b-b0be-a1aa51c2a6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:35.682516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9a2fc-dd64-442b-b0be-a1aa51c2a6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:35.682782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9a2fc-dd64-442b-b0be-a1aa51c2a6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:35.683254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9a2fc-dd64-442b-b0be-a1aa51c2a6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:35.684148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9a2fc-dd64-442b-b0be-a1aa51c2a6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:35.688075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9a2fc-dd64-442b-b0be-a1aa51c2a6f3 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:35.704948 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1326/2621] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:46:35.706605 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbc835-8bb7-472f-a5dc-eedc748e95f7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:35.717697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbc835-8bb7-472f-a5dc-eedc748e95f7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:35.722951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c394e-cdae-4fb2-aae4-ae5e17497cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:35.724168 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c394e-cdae-4fb2-aae4-ae5e17497cec admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:35.727393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c394e-cdae-4fb2-aae4-ae5e17497cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:35.727393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c394e-cdae-4fb2-aae4-ae5e17497cec admin admin] PATH_INFO: `/v3/projects/baa4fb911d194621bcf35c84c846ea9c/users/4c8657afe9604c20a4c92f8195adcb1c/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:35.750854 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c394e-cdae-4fb2-aae4-ae5e17497cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa4fb911d194621bcf35c84c846ea9c, user_id=4c8657afe9604c20a4c92f8195adcb1c, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:35.754449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c394e-cdae-4fb2-aae4-ae5e17497cec admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:35.847814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1296/2622] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:35 2022] PUT /identity/v3/projects/baa4fb911d194621bcf35c84c846ea9c/users/4c8657afe9604c20a4c92f8195adcb1c/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:35.856622 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9a2fc-dd64-442b-b0be-a1aa51c2a6f3 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:35.864762 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9a2fc-dd64-442b-b0be-a1aa51c2a6f3 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:35.934649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:35.935411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:35.935693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:35.935966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:35.936658 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:35.938253 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:35.953055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1327/2623] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:35.954317 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c394e-cdae-4fb2-aae4-ae5e17497cec admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:35.968839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c394e-cdae-4fb2-aae4-ae5e17497cec admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:35.972218 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583c88f-dd6d-480d-95a7-539306543c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:35.972933 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c88f-dd6d-480d-95a7-539306543c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:35.973225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c88f-dd6d-480d-95a7-539306543c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:35.974139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583c88f-dd6d-480d-95a7-539306543c8a admin admin] PATH_INFO: `/v3/projects/baa4fb911d194621bcf35c84c846ea9c/users/4c8657afe9604c20a4c92f8195adcb1c/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:35.976308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583c88f-dd6d-480d-95a7-539306543c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa4fb911d194621bcf35c84c846ea9c, user_id=4c8657afe9604c20a4c92f8195adcb1c, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:35.979012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583c88f-dd6d-480d-95a7-539306543c8a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:36.001721 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1297/2624] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:35 2022] PUT /identity/v3/projects/baa4fb911d194621bcf35c84c846ea9c/users/4c8657afe9604c20a4c92f8195adcb1c/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:38.739076 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:38.741198 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:38.762949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df21aed-0cd9-4fc9-8583-a3358dba0234 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:38.898129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:38.900519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:38.900519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:38.901016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] PATH_INFO: `/v3/users/eeafd7e29adf43f2a4849950931a645a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:38.903473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeafd7e29adf43f2a4849950931a645a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:38.906966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:38.919463 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:39.218326 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeafd7e29adf43f2a4849950931a645a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:39.219659 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeafd7e29adf43f2a4849950931a645a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:39.233745 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeafd7e29adf43f2a4849950931a645a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:39.382502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeafd7e29adf43f2a4849950931a645a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:39.383240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeafd7e29adf43f2a4849950931a645a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:39.422160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1328/2625] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:38 2022] DELETE /identity/v3/users/eeafd7e29adf43f2a4849950931a645a => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:39.429839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5ff72-468e-4cf2-a18a-abdda4c9428e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:39.430656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5ff72-468e-4cf2-a18a-abdda4c9428e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:39.430656 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5ff72-468e-4cf2-a18a-abdda4c9428e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:39.465488 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-a3a5ff72-468e-4cf2-a18a-abdda4c9428e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:39.466947 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a3a5ff72-468e-4cf2-a18a-abdda4c9428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:39.547121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:39.569091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1298/2626] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:39.576050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:39.577603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:39.683154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb845b-bf24-4478-818c-fd3c4f4b5262 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:39.688016 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e22a7f-10e3-4187-9688-95b74382410f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:39.689003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e22a7f-10e3-4187-9688-95b74382410f service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:39.689468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e22a7f-10e3-4187-9688-95b74382410f service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:39.690159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e22a7f-10e3-4187-9688-95b74382410f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:39.704983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e22a7f-10e3-4187-9688-95b74382410f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:39.706905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e22a7f-10e3-4187-9688-95b74382410f service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:39.720006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1329/2627] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:40.977027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5ff72-468e-4cf2-a18a-abdda4c9428e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:40.977027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a3a5ff72-468e-4cf2-a18a-abdda4c9428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:40.997931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5ff72-468e-4cf2-a18a-abdda4c9428e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:41.006469 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:41.007368 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:41.008011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:41.008727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] PATH_INFO: `/v3/projects/f7357cf5935541a7aba0fb7c46e224cc` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:41.011086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7357cf5935541a7aba0fb7c46e224cc)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:41.015442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:41.266930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7357cf5935541a7aba0fb7c46e224cc', 'request_id': 'req-c9529584-c889-4fb3-b133-051dab0c7863'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:41.299892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7357cf5935541a7aba0fb7c46e224cc', 'request_id': 'req-c9529584-c889-4fb3-b133-051dab0c7863'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:41.313569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7357cf5935541a7aba0fb7c46e224cc', 'request_id': 'req-c9529584-c889-4fb3-b133-051dab0c7863'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:41.433272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] The token cache is being invalidate because project f7357cf5935541a7aba0fb7c46e224cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:41.433978 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:41.438134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:41.440581 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1299/2628] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:40 2022] DELETE /identity/v3/projects/f7357cf5935541a7aba0fb7c46e224cc => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:41.444542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e22a7f-10e3-4187-9688-95b74382410f service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:41.455739 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e22a7f-10e3-4187-9688-95b74382410f service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:41.532702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:41.533227 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:41.533516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:41.533789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] PATH_INFO: `/v3/users/f8a6746337dc46dabd71eca90cadecc1` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:41.537995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8a6746337dc46dabd71eca90cadecc1)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:41.543018 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:41.557491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:41.947811 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8a6746337dc46dabd71eca90cadecc1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:41.948721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8a6746337dc46dabd71eca90cadecc1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:41.957715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8a6746337dc46dabd71eca90cadecc1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:42.159302 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8a6746337dc46dabd71eca90cadecc1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:42.160200 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9afa0fb4-db09-434e-9470-cdda560e581d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8a6746337dc46dabd71eca90cadecc1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:42.180642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1330/2629] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:41 2022] DELETE /identity/v3/users/f8a6746337dc46dabd71eca90cadecc1 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:43.155515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:43.158145 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:43.173866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9529584-c889-4fb3-b133-051dab0c7863 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:43.266685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:43.267439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:43.267724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:43.268389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] PATH_INFO: `/v3/projects/90bbc679faf847fe9ef79f7cea247d94` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:43.269589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90bbc679faf847fe9ef79f7cea247d94)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:43.272066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:43.432564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90bbc679faf847fe9ef79f7cea247d94', 'request_id': 'req-ade3c461-486b-428d-adc8-10d3ddf159d5'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:43.439457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90bbc679faf847fe9ef79f7cea247d94', 'request_id': 'req-ade3c461-486b-428d-adc8-10d3ddf159d5'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:43.448265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90bbc679faf847fe9ef79f7cea247d94', 'request_id': 'req-ade3c461-486b-428d-adc8-10d3ddf159d5'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:43.565099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] The token cache is being invalidate because project 90bbc679faf847fe9ef79f7cea247d94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:43.565489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:43.566315 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ade3c461-486b-428d-adc8-10d3ddf159d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:43.569773 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1300/2630] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:43 2022] DELETE /identity/v3/projects/90bbc679faf847fe9ef79f7cea247d94 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:43.575711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e036-6cd4-42f5-945b-29b40d6c6427 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:43.576155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e036-6cd4-42f5-945b-29b40d6c6427 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:43.576348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e036-6cd4-42f5-945b-29b40d6c6427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:43.591303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15af50-2f80-4ffd-b7ca-debc678b34c7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:43.591586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15af50-2f80-4ffd-b7ca-debc678b34c7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:43.591861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15af50-2f80-4ffd-b7ca-debc678b34c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:43.597515 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-af08e036-6cd4-42f5-945b-29b40d6c6427 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:43.599846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-af08e036-6cd4-42f5-945b-29b40d6c6427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:43.606796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:43.623918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6c15af50-2f80-4ffd-b7ca-debc678b34c7 None None] MFA Rules not processed for user `4c8657afe9604c20a4c92f8195adcb1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:43.626395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6c15af50-2f80-4ffd-b7ca-debc678b34c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:43.682977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1331/2631] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:43.692807 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4486cf5-ca3f-4acb-b12d-4bdf5b8cd25d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:43.693098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4486cf5-ca3f-4acb-b12d-4bdf5b8cd25d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:43.693379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4486cf5-ca3f-4acb-b12d-4bdf5b8cd25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:43.711992 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1301/2632] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:43.714740 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-f4486cf5-ca3f-4acb-b12d-4bdf5b8cd25d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:43.715972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-f4486cf5-ca3f-4acb-b12d-4bdf5b8cd25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:43.725419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15af50-2f80-4ffd-b7ca-debc678b34c7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:43.726643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1332/2633] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:43.728406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6c15af50-2f80-4ffd-b7ca-debc678b34c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:43.730943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4486cf5-ca3f-4acb-b12d-4bdf5b8cd25d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:43.740383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4486cf5-ca3f-4acb-b12d-4bdf5b8cd25d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:43.745732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:43.746292 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:43.746761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:43.747012 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:43.747556 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:46:43.747819 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:43.750176 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:43.763548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1333/2634] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 13:46:43.774184 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:43.784087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610873d3-4ab2-46ea-8eff-fd28c8e5abf7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:43.789154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7fd92-87dd-4493-90fe-182e6279997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:43.789756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7fd92-87dd-4493-90fe-182e6279997e admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:43.790090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7fd92-87dd-4493-90fe-182e6279997e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:43.790367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7fd92-87dd-4493-90fe-182e6279997e admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:43.791038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7fd92-87dd-4493-90fe-182e6279997e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:43.792707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15af50-2f80-4ffd-b7ca-debc678b34c7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:43.793138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7fd92-87dd-4493-90fe-182e6279997e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:43.797113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5c6fcfde34c05a2fec0659dfd54a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:43.797828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:43.798174 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:43.798450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:43.800306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:43.866064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:43.867447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:43.877956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1302/2635] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:46:43.949896 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1334/2636] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:43 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:43.953943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:43.965829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca491589-9323-46cc-a747-8b5e9947e16b service glance] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:44.022350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb30643-066e-4dcc-a71b-320bd18f1be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:44.022962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30643-066e-4dcc-a71b-320bd18f1be3 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.023245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30643-066e-4dcc-a71b-320bd18f1be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.023519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30643-066e-4dcc-a71b-320bd18f1be3 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.024147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb30643-066e-4dcc-a71b-320bd18f1be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:44.026177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb30643-066e-4dcc-a71b-320bd18f1be3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:44.134361 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1303/2637] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:43 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:44.135519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7fd92-87dd-4493-90fe-182e6279997e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:44.149052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7fd92-87dd-4493-90fe-182e6279997e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:44.153831 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddc854-9b8b-41c4-b57d-9e808fac4b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:44.154495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddc854-9b8b-41c4-b57d-9e808fac4b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.154750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddc854-9b8b-41c4-b57d-9e808fac4b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.154990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddc854-9b8b-41c4-b57d-9e808fac4b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.155639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ddc854-9b8b-41c4-b57d-9e808fac4b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:44.157272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ddc854-9b8b-41c4-b57d-9e808fac4b97 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:44.168301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1335/2638] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:44.172546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb30643-066e-4dcc-a71b-320bd18f1be3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:44.184026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb30643-066e-4dcc-a71b-320bd18f1be3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:44.189120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf229b-cd05-447b-8ea2-8455c710a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:44.190002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf229b-cd05-447b-8ea2-8455c710a4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.190295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf229b-cd05-447b-8ea2-8455c710a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.190608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf229b-cd05-447b-8ea2-8455c710a4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.191222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf229b-cd05-447b-8ea2-8455c710a4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:44.193960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf229b-cd05-447b-8ea2-8455c710a4e0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:44.211135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1304/2639] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:44.211483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddc854-9b8b-41c4-b57d-9e808fac4b97 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:44.222551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d20200-d78e-45fd-9f4f-10432058ebb0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.224092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d20200-d78e-45fd-9f4f-10432058ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.224092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d20200-d78e-45fd-9f4f-10432058ebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.229803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddc854-9b8b-41c4-b57d-9e808fac4b97 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:44.237990 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:44.239718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.240235 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.240969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] PATH_INFO: `/v3/projects/4f3fa76e7f7c460dbf09ff7d8186d0c0/users/4de6d4948c9645fcb4a4a2e2e64a1096/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.257007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-f8d20200-d78e-45fd-9f4f-10432058ebb0 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:44.262262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f8d20200-d78e-45fd-9f4f-10432058ebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:44.269846 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3fa76e7f7c460dbf09ff7d8186d0c0, user_id=4de6d4948c9645fcb4a4a2e2e64a1096, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:44.272750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:44.341920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1305/2640] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:46:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:44.436264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1336/2641] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:44 2022] PUT /identity/v3/projects/4f3fa76e7f7c460dbf09ff7d8186d0c0/users/4de6d4948c9645fcb4a4a2e2e64a1096/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:44.447652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62105fe5-c61e-4418-92ba-f2ba8114815b None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.447940 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62105fe5-c61e-4418-92ba-f2ba8114815b None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.448382 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62105fe5-c61e-4418-92ba-f2ba8114815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.480284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-62105fe5-c61e-4418-92ba-f2ba8114815b None None] MFA Rules not processed for user `4de6d4948c9645fcb4a4a2e2e64a1096`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:44.481593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-62105fe5-c61e-4418-92ba-f2ba8114815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:44.546787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:44.548637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:44.558037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1306/2642] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:44.562594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62105fe5-c61e-4418-92ba-f2ba8114815b None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:44.567668 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:44.594033 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:44.636616 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65295503-7821-47a4-ac02-abd3287d8e89 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:44.643216 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c231beb-441a-47bd-96da-d39a2c15231a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:44.643835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c231beb-441a-47bd-96da-d39a2c15231a service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.644120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c231beb-441a-47bd-96da-d39a2c15231a service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.644432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c231beb-441a-47bd-96da-d39a2c15231a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.648648 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62105fe5-c61e-4418-92ba-f2ba8114815b None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:44.651272 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b940e-440a-465c-ade1-7703305bd529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:44.651879 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b940e-440a-465c-ade1-7703305bd529 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.652158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b940e-440a-465c-ade1-7703305bd529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.653713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b940e-440a-465c-ade1-7703305bd529 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.653713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b940e-440a-465c-ade1-7703305bd529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:44.655513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b940e-440a-465c-ade1-7703305bd529 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:44.701727 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:44.706371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c231beb-441a-47bd-96da-d39a2c15231a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:44.708113 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c231beb-441a-47bd-96da-d39a2c15231a service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:44.716747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1337/2643] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:46:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 13:46:44.845104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c231beb-441a-47bd-96da-d39a2c15231a service swift] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:44.849049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1307/2644] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:44 2022] POST /identity/v3/projects => generated 385 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:44.863988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c231beb-441a-47bd-96da-d39a2c15231a service swift] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:44.947294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaa597-4013-4e74-ba0e-c99a4c941e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:44.948151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa597-4013-4e74-ba0e-c99a4c941e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:44.950765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa597-4013-4e74-ba0e-c99a4c941e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:44.956750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa597-4013-4e74-ba0e-c99a4c941e0c admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:44.956750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaa597-4013-4e74-ba0e-c99a4c941e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:44.956750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaa597-4013-4e74-ba0e-c99a4c941e0c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:45.096692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1338/2645] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:44 2022] POST /identity/v3/users => generated 299 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:45.098805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b940e-440a-465c-ade1-7703305bd529 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:45.113471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b940e-440a-465c-ade1-7703305bd529 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:45.122524 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a748e-04c6-43a0-b61c-b5f7afa836d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:45.123239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a748e-04c6-43a0-b61c-b5f7afa836d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:45.123703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a748e-04c6-43a0-b61c-b5f7afa836d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:45.124029 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a748e-04c6-43a0-b61c-b5f7afa836d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:45.124996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a748e-04c6-43a0-b61c-b5f7afa836d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:45.136383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a748e-04c6-43a0-b61c-b5f7afa836d3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:45.163134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1308/2646] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:45.167375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaa597-4013-4e74-ba0e-c99a4c941e0c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:45.182492 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaa597-4013-4e74-ba0e-c99a4c941e0c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:45.189438 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee628f3-67fd-4d82-a417-7e3d3ec56feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:45.190666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee628f3-67fd-4d82-a417-7e3d3ec56feb admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:45.191024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee628f3-67fd-4d82-a417-7e3d3ec56feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:45.191426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee628f3-67fd-4d82-a417-7e3d3ec56feb admin admin] PATH_INFO: `/v3/projects/7b3abb08cadf4a34bdaf7c4ea7eb5bfb/users/22e3d05b06be470e8c5f0e9ff47189b4/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:45.259090 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee628f3-67fd-4d82-a417-7e3d3ec56feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3abb08cadf4a34bdaf7c4ea7eb5bfb, user_id=22e3d05b06be470e8c5f0e9ff47189b4, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:45.263098 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee628f3-67fd-4d82-a417-7e3d3ec56feb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:45.352316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1339/2647] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:45 2022] PUT /identity/v3/projects/7b3abb08cadf4a34bdaf7c4ea7eb5bfb/users/22e3d05b06be470e8c5f0e9ff47189b4/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:45.356393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a748e-04c6-43a0-b61c-b5f7afa836d3 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:45.377325 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a748e-04c6-43a0-b61c-b5f7afa836d3 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:45.484568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f2cdc-d74e-4a3f-9eb3-4427832f9a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:45.485277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f2cdc-d74e-4a3f-9eb3-4427832f9a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:45.485542 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f2cdc-d74e-4a3f-9eb3-4427832f9a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:45.485814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f2cdc-d74e-4a3f-9eb3-4427832f9a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:45.486458 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f2cdc-d74e-4a3f-9eb3-4427832f9a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:45.488980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f2cdc-d74e-4a3f-9eb3-4427832f9a0d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:45.508736 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1309/2648] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:45.510020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee628f3-67fd-4d82-a417-7e3d3ec56feb admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:45.525214 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee628f3-67fd-4d82-a417-7e3d3ec56feb admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:45.530383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:45.531077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:45.531635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:45.531917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] PATH_INFO: `/v3/projects/7b3abb08cadf4a34bdaf7c4ea7eb5bfb/users/22e3d05b06be470e8c5f0e9ff47189b4/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:45.534915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3abb08cadf4a34bdaf7c4ea7eb5bfb, user_id=22e3d05b06be470e8c5f0e9ff47189b4, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:45.538281 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:45.559283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1340/2649] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:45 2022] PUT /identity/v3/projects/7b3abb08cadf4a34bdaf7c4ea7eb5bfb/users/22e3d05b06be470e8c5f0e9ff47189b4/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:45.573125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172e38a-a1d2-41b3-bb1e-651e3adf9b34 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:45.573125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172e38a-a1d2-41b3-bb1e-651e3adf9b34 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:45.573125 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172e38a-a1d2-41b3-bb1e-651e3adf9b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:45.612271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-8172e38a-a1d2-41b3-bb1e-651e3adf9b34 None None] MFA Rules not processed for user `22e3d05b06be470e8c5f0e9ff47189b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:45.614467 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-8172e38a-a1d2-41b3-bb1e-651e3adf9b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:45.735873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1310/2650] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:45.748419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:45.752472 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:45.760335 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:45.862856 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e790cc-8087-4d31-9256-64c95d82a87a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:45.866823 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47284e-22a5-4269-a54a-d3aebab130b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:45.867439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47284e-22a5-4269-a54a-d3aebab130b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:45.867718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47284e-22a5-4269-a54a-d3aebab130b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:45.867991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47284e-22a5-4269-a54a-d3aebab130b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:45.882637 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47284e-22a5-4269-a54a-d3aebab130b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:45.884300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47284e-22a5-4269-a54a-d3aebab130b9 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:45.897164 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1341/2651] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:45.932843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172e38a-a1d2-41b3-bb1e-651e3adf9b34 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:45.998649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:46.022011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172e38a-a1d2-41b3-bb1e-651e3adf9b34 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:46.028232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:46.029016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:46.029425 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:46.029795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] PATH_INFO: `/v3/users/4de6d4948c9645fcb4a4a2e2e64a1096` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:46.031569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de6d4948c9645fcb4a4a2e2e64a1096)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:46.034121 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:46.245903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de6d4948c9645fcb4a4a2e2e64a1096'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:46.246748 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de6d4948c9645fcb4a4a2e2e64a1096'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:46.271163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de6d4948c9645fcb4a4a2e2e64a1096'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:46.273995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de6d4948c9645fcb4a4a2e2e64a1096'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:46.375952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de6d4948c9645fcb4a4a2e2e64a1096'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:46.415512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1311/2652] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:45 2022] DELETE /identity/v3/users/4de6d4948c9645fcb4a4a2e2e64a1096 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:46.420537 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4617d7-1d70-4960-a127-535369dfa5b4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:46.421525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4617d7-1d70-4960-a127-535369dfa5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:46.422329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4617d7-1d70-4960-a127-535369dfa5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:46.469841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-cc4617d7-1d70-4960-a127-535369dfa5b4 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:46.472470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cc4617d7-1d70-4960-a127-535369dfa5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:46.556070 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1342/2653] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:46.569201 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:46.572946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:46.711725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d209e9-3e49-4600-b76f-7d8ae5ccbdb4 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:46.716088 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cdad8-5fa1-4eb2-96dc-a0a8cdfe749c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:46.716738 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cdad8-5fa1-4eb2-96dc-a0a8cdfe749c service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:46.717071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cdad8-5fa1-4eb2-96dc-a0a8cdfe749c service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:46.717491 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cdad8-5fa1-4eb2-96dc-a0a8cdfe749c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:46.733741 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cdad8-5fa1-4eb2-96dc-a0a8cdfe749c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:46.735511 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cdad8-5fa1-4eb2-96dc-a0a8cdfe749c service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:46.760752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1312/2654] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:47.416848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4617d7-1d70-4960-a127-535369dfa5b4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:47.417953 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-cc4617d7-1d70-4960-a127-535369dfa5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:47.443542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4617d7-1d70-4960-a127-535369dfa5b4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:47.452777 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:47.454027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:47.454348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:47.454696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] PATH_INFO: `/v3/projects/4f3fa76e7f7c460dbf09ff7d8186d0c0` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:47.455918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3fa76e7f7c460dbf09ff7d8186d0c0)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:47.458400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:47.636915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3fa76e7f7c460dbf09ff7d8186d0c0', 'request_id': 'req-e0981d46-3dd3-4809-88ec-ba5b574985a5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:47.650040 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3fa76e7f7c460dbf09ff7d8186d0c0', 'request_id': 'req-e0981d46-3dd3-4809-88ec-ba5b574985a5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:47.736241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3fa76e7f7c460dbf09ff7d8186d0c0', 'request_id': 'req-e0981d46-3dd3-4809-88ec-ba5b574985a5'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:47.757797 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] The token cache is being invalidate because project 4f3fa76e7f7c460dbf09ff7d8186d0c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:47.758482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:47.759645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:47.762005 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1343/2655] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:47 2022] DELETE /identity/v3/projects/4f3fa76e7f7c460dbf09ff7d8186d0c0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:47.765934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cdad8-5fa1-4eb2-96dc-a0a8cdfe749c service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:47.777400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cdad8-5fa1-4eb2-96dc-a0a8cdfe749c service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:47.859595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:47.860479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:47.860887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:47.861411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] PATH_INFO: `/v3/users/22e3d05b06be470e8c5f0e9ff47189b4` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:47.863888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e3d05b06be470e8c5f0e9ff47189b4)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:47.865812 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:47.882311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:48.323528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e3d05b06be470e8c5f0e9ff47189b4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:48.324222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e3d05b06be470e8c5f0e9ff47189b4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:48.337957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e3d05b06be470e8c5f0e9ff47189b4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:48.338711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e3d05b06be470e8c5f0e9ff47189b4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:48.398815 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6bc63bed-4cec-48a0-8928-9838021a6f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e3d05b06be470e8c5f0e9ff47189b4'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:48.431096 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1313/2656] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:47 2022] DELETE /identity/v3/users/22e3d05b06be470e8c5f0e9ff47189b4 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:49.070478 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:49.073231 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:49.107063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0981d46-3dd3-4809-88ec-ba5b574985a5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:49.131962 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:49.229731 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:49.230386 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:49.230671 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:49.230965 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] PATH_INFO: `/v3/projects/7b3abb08cadf4a34bdaf7c4ea7eb5bfb` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:49.235092 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3abb08cadf4a34bdaf7c4ea7eb5bfb)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:49.237722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:49.397002 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3abb08cadf4a34bdaf7c4ea7eb5bfb', 'request_id': 'req-2455b5e0-58b5-4027-a0f0-919edbe05efe'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:49.404958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3abb08cadf4a34bdaf7c4ea7eb5bfb', 'request_id': 'req-2455b5e0-58b5-4027-a0f0-919edbe05efe'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:49.490000 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3abb08cadf4a34bdaf7c4ea7eb5bfb', 'request_id': 'req-2455b5e0-58b5-4027-a0f0-919edbe05efe'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:49.503652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] The token cache is being invalidate because project 7b3abb08cadf4a34bdaf7c4ea7eb5bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:49.504063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:49.504972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2455b5e0-58b5-4027-a0f0-919edbe05efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:49.506714 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1344/2657] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:49 2022] DELETE /identity/v3/projects/7b3abb08cadf4a34bdaf7c4ea7eb5bfb => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:49.511746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bac013-323f-41f4-8479-0f1d599be656 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:49.512284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bac013-323f-41f4-8479-0f1d599be656 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:49.512503 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bac013-323f-41f4-8479-0f1d599be656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:49.534183 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-62bac013-323f-41f4-8479-0f1d599be656 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:49.536171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-62bac013-323f-41f4-8479-0f1d599be656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:49.636718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1314/2658] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:49.647105 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55520ce3-b5f2-4719-9c40-5291722f2eaa None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:49.647732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55520ce3-b5f2-4719-9c40-5291722f2eaa None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:49.648346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55520ce3-b5f2-4719-9c40-5291722f2eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:49.683305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-55520ce3-b5f2-4719-9c40-5291722f2eaa None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:49.684927 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-55520ce3-b5f2-4719-9c40-5291722f2eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:49.695887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1345/2659] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:49.700849 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bac013-323f-41f4-8479-0f1d599be656 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:49.713840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bac013-323f-41f4-8479-0f1d599be656 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:49.719949 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:49.720696 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:49.721053 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:49.721423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:49.722159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:46:49.722667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:49.725747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:49.746675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55520ce3-b5f2-4719-9c40-5291722f2eaa None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:49.753861 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1315/2660] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:46:49.758424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55520ce3-b5f2-4719-9c40-5291722f2eaa None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:49.765066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227144d-afb8-4342-91ca-4f7a76b96043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:49.765956 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227144d-afb8-4342-91ca-4f7a76b96043 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:49.768385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227144d-afb8-4342-91ca-4f7a76b96043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:49.768769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227144d-afb8-4342-91ca-4f7a76b96043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:49.769539 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227144d-afb8-4342-91ca-4f7a76b96043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:49.772401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227144d-afb8-4342-91ca-4f7a76b96043 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:49.872848 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1346/2661] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:49 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:49.874358 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:49.886755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80278566-cf0b-411f-afed-ae7c99ffede2 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:49.965820 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f3d1e-f04c-4558-9fc3-c2a8588f75ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:49.966440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3d1e-f04c-4558-9fc3-c2a8588f75ff admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:49.966707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3d1e-f04c-4558-9fc3-c2a8588f75ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:49.967006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3d1e-f04c-4558-9fc3-c2a8588f75ff admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:49.967632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f3d1e-f04c-4558-9fc3-c2a8588f75ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:49.969644 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f3d1e-f04c-4558-9fc3-c2a8588f75ff admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:50.077709 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1316/2662] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:49 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:50.081645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227144d-afb8-4342-91ca-4f7a76b96043 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:50.099685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227144d-afb8-4342-91ca-4f7a76b96043 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:50.105019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6205a-e772-45d9-a9d5-f2ec3660007d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:50.105646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6205a-e772-45d9-a9d5-f2ec3660007d admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:50.109047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6205a-e772-45d9-a9d5-f2ec3660007d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:50.109344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6205a-e772-45d9-a9d5-f2ec3660007d admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:50.110062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6205a-e772-45d9-a9d5-f2ec3660007d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:50.112033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6205a-e772-45d9-a9d5-f2ec3660007d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:50.125083 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1347/2663] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:50.132154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f3d1e-f04c-4558-9fc3-c2a8588f75ff admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:50.139008 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:50.143840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f3d1e-f04c-4558-9fc3-c2a8588f75ff admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:50.152220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03827c57-f40a-44fd-8629-6d4f823dafc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:50.153036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03827c57-f40a-44fd-8629-6d4f823dafc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:50.153419 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03827c57-f40a-44fd-8629-6d4f823dafc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:50.153805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03827c57-f40a-44fd-8629-6d4f823dafc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:50.154566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03827c57-f40a-44fd-8629-6d4f823dafc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:50.157070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03827c57-f40a-44fd-8629-6d4f823dafc0 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:50.172719 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1317/2664] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:50.178898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6205a-e772-45d9-a9d5-f2ec3660007d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:50.198826 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6205a-e772-45d9-a9d5-f2ec3660007d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:50.203988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:50.204782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:50.205160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:50.205614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] PATH_INFO: `/v3/projects/09a4b0ec7caf40878190f9690eb0bb0d/users/6f688b2bf4cc49e2b42f80711e6424ae/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:50.228115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a4b0ec7caf40878190f9690eb0bb0d, user_id=6f688b2bf4cc49e2b42f80711e6424ae, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:50.233336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:50.300134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1348/2665] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:50 2022] PUT /identity/v3/projects/09a4b0ec7caf40878190f9690eb0bb0d/users/6f688b2bf4cc49e2b42f80711e6424ae/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:50.321641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03281477-022a-45ee-bf3f-390a7950b440 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:50.323109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03281477-022a-45ee-bf3f-390a7950b440 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:50.323635 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03281477-022a-45ee-bf3f-390a7950b440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:50.369767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-03281477-022a-45ee-bf3f-390a7950b440 None None] MFA Rules not processed for user `6f688b2bf4cc49e2b42f80711e6424ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:50.371226 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-03281477-022a-45ee-bf3f-390a7950b440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:50.455403 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1318/2666] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:46:50.469062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:50.471431 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:50.565865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:50.595385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd48b-627f-4250-9980-55e0afd51454 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:50.604485 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d5c18-1088-4c26-bbef-49ad52d7a4ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:50.605730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d5c18-1088-4c26-bbef-49ad52d7a4ac service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:50.606086 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d5c18-1088-4c26-bbef-49ad52d7a4ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:50.609503 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d5c18-1088-4c26-bbef-49ad52d7a4ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:50.623024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d5c18-1088-4c26-bbef-49ad52d7a4ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:50.624533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d5c18-1088-4c26-bbef-49ad52d7a4ac service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:50.637606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1349/2667] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:50.667905 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03281477-022a-45ee-bf3f-390a7950b440 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:50.773437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03281477-022a-45ee-bf3f-390a7950b440 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:50.780258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:50.785979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:50.785979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:50.785979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] PATH_INFO: `/v3/users/6f688b2bf4cc49e2b42f80711e6424ae` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:50.785979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f688b2bf4cc49e2b42f80711e6424ae)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:50.789805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:51.018089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f688b2bf4cc49e2b42f80711e6424ae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:51.018770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f688b2bf4cc49e2b42f80711e6424ae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:51.033480 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f688b2bf4cc49e2b42f80711e6424ae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:51.034243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f688b2bf4cc49e2b42f80711e6424ae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:51.095878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f688b2bf4cc49e2b42f80711e6424ae'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:51.122585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1319/2668] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:46:50 2022] DELETE /identity/v3/users/6f688b2bf4cc49e2b42f80711e6424ae => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:51.127570 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef74cc7-3e3b-440b-a567-efc4223a0bb8 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:51.128280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef74cc7-3e3b-440b-a567-efc4223a0bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:51.128280 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef74cc7-3e3b-440b-a567-efc4223a0bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:51.151284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-aef74cc7-3e3b-440b-a567-efc4223a0bb8 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:51.153530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-aef74cc7-3e3b-440b-a567-efc4223a0bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:51.323635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1350/2669] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:51.333147 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:51.334585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:51.339979 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:51.403271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17db5c9-0dc3-4828-8cb1-04918e997d69 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:51.407141 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0575b-5168-4908-941c-9c7159d53703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:51.407826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0575b-5168-4908-941c-9c7159d53703 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:51.408129 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0575b-5168-4908-941c-9c7159d53703 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:51.408450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0575b-5168-4908-941c-9c7159d53703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:51.418866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0575b-5168-4908-941c-9c7159d53703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:51.420338 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0575b-5168-4908-941c-9c7159d53703 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:51.431645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1320/2670] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:51.930291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef74cc7-3e3b-440b-a567-efc4223a0bb8 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:51.932121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-aef74cc7-3e3b-440b-a567-efc4223a0bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:51.953951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef74cc7-3e3b-440b-a567-efc4223a0bb8 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:51.962606 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:51.963277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:51.965366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:51.965366 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] PATH_INFO: `/v3/projects/09a4b0ec7caf40878190f9690eb0bb0d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:51.965922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a4b0ec7caf40878190f9690eb0bb0d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:51.969530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:52.207194 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a4b0ec7caf40878190f9690eb0bb0d', 'request_id': 'req-e406e9a4-38be-4698-a8ed-37f26ebab68e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:52.216131 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a4b0ec7caf40878190f9690eb0bb0d', 'request_id': 'req-e406e9a4-38be-4698-a8ed-37f26ebab68e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:52.302089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a4b0ec7caf40878190f9690eb0bb0d', 'request_id': 'req-e406e9a4-38be-4698-a8ed-37f26ebab68e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:52.321197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] The token cache is being invalidate because project 09a4b0ec7caf40878190f9690eb0bb0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:46:52.321583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:52.322522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-e406e9a4-38be-4698-a8ed-37f26ebab68e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:46:52.329114 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1351/2671] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:46:51 2022] DELETE /identity/v3/projects/09a4b0ec7caf40878190f9690eb0bb0d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:46:52.329935 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ba808-1239-46ec-8b6f-490507fc15a0 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:52.330744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ba808-1239-46ec-8b6f-490507fc15a0 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:52.331229 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ba808-1239-46ec-8b6f-490507fc15a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:52.358413 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-953ba808-1239-46ec-8b6f-490507fc15a0 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:52.359685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-953ba808-1239-46ec-8b6f-490507fc15a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:52.448411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1321/2672] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:52.457875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa6fc-dca5-4bb9-9260-26547ba53966 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:52.458192 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa6fc-dca5-4bb9-9260-26547ba53966 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:52.458563 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffa6fc-dca5-4bb9-9260-26547ba53966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:52.483812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-30ffa6fc-dca5-4bb9-9260-26547ba53966 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:52.485111 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-30ffa6fc-dca5-4bb9-9260-26547ba53966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:52.494723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1352/2673] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:52.498929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ba808-1239-46ec-8b6f-490507fc15a0 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:52.511899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ba808-1239-46ec-8b6f-490507fc15a0 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:52.517875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:52.518625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:52.519143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:52.520211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:52.521123 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:46:52.521417 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:52.523746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:52.544119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1322/2674] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:46:52.544434 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffa6fc-dca5-4bb9-9260-26547ba53966 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:52.556902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffa6fc-dca5-4bb9-9260-26547ba53966 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:52.562159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032bc9c-ac44-4e06-a914-e915aa840c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:52.563001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032bc9c-ac44-4e06-a914-e915aa840c95 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:52.563403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032bc9c-ac44-4e06-a914-e915aa840c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:52.563787 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032bc9c-ac44-4e06-a914-e915aa840c95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:52.564538 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032bc9c-ac44-4e06-a914-e915aa840c95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:52.567091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032bc9c-ac44-4e06-a914-e915aa840c95 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:52.714181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1353/2675] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:46:52 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:46:52.715374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:52.736917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78819b-0f41-494f-940c-b388ccfc1de7 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:52.811509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78db29-c728-4287-93b1-dfcf52f076e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:52.812118 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78db29-c728-4287-93b1-dfcf52f076e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:52.812438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78db29-c728-4287-93b1-dfcf52f076e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:52.812712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78db29-c728-4287-93b1-dfcf52f076e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:52.813344 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78db29-c728-4287-93b1-dfcf52f076e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:52.815568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78db29-c728-4287-93b1-dfcf52f076e2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:52.916019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1323/2676] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:46:52 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:46:52.920308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032bc9c-ac44-4e06-a914-e915aa840c95 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:52.931558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032bc9c-ac44-4e06-a914-e915aa840c95 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:52.938869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cad86e-226b-4708-870f-4d0aaac4b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:52.939615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cad86e-226b-4708-870f-4d0aaac4b97b admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:52.940251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cad86e-226b-4708-870f-4d0aaac4b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:52.940599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cad86e-226b-4708-870f-4d0aaac4b97b admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:52.941321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cad86e-226b-4708-870f-4d0aaac4b97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:52.944068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cad86e-226b-4708-870f-4d0aaac4b97b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:52.957165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1354/2677] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:52.962409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78db29-c728-4287-93b1-dfcf52f076e2 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:52.972617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78db29-c728-4287-93b1-dfcf52f076e2 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:52.979642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e989d-4865-4964-8db5-8b193fcbf33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:52.980402 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e989d-4865-4964-8db5-8b193fcbf33c admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:52.980778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e989d-4865-4964-8db5-8b193fcbf33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:52.981181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e989d-4865-4964-8db5-8b193fcbf33c admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:52.981914 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e989d-4865-4964-8db5-8b193fcbf33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:52.983660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e989d-4865-4964-8db5-8b193fcbf33c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:52.996636 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1324/2678] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:46:53.000957 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cad86e-226b-4708-870f-4d0aaac4b97b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:53.012718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cad86e-226b-4708-870f-4d0aaac4b97b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:53.017761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:53.018513 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:53.018884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:53.019329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] PATH_INFO: `/v3/projects/face821ff8c7474ebdc02058b43dc575/users/38f8522facb34b9b9660136e4dd214f2/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:53.050001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=face821ff8c7474ebdc02058b43dc575, user_id=38f8522facb34b9b9660136e4dd214f2, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:53.053749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:53.119875 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:53.125213 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1355/2679] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:46:52 2022] PUT /identity/v3/projects/face821ff8c7474ebdc02058b43dc575/users/38f8522facb34b9b9660136e4dd214f2/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:46:53.136627 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ee5e0-a92b-4bd8-8c9c-548f112b3054 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:53.136899 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ee5e0-a92b-4bd8-8c9c-548f112b3054 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:53.137178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ee5e0-a92b-4bd8-8c9c-548f112b3054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:53.163231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:53.175406 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e35ee5e0-a92b-4bd8-8c9c-548f112b3054 None None] MFA Rules not processed for user `38f8522facb34b9b9660136e4dd214f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:53.177273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e35ee5e0-a92b-4bd8-8c9c-548f112b3054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:53.261143 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1325/2680] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:53.275931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:53.278003 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:53.352603 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db53ac-4ea5-4cbc-af65-683682b6eb8d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:53.358819 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783a12b-7e04-4b9a-8eb9-8945d85fe0b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5c6fcfde34c05a2fec0659dfd54a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:53.359444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783a12b-7e04-4b9a-8eb9-8945d85fe0b3 service glance] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:53.359721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783a12b-7e04-4b9a-8eb9-8945d85fe0b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:53.359991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783a12b-7e04-4b9a-8eb9-8945d85fe0b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:53.370460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783a12b-7e04-4b9a-8eb9-8945d85fe0b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:53.371926 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783a12b-7e04-4b9a-8eb9-8945d85fe0b3 service glance] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:53.385718 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1356/2681] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:46:54.378775 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ee5e0-a92b-4bd8-8c9c-548f112b3054 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:54.380566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e35ee5e0-a92b-4bd8-8c9c-548f112b3054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:54.452499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ee5e0-a92b-4bd8-8c9c-548f112b3054 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:54.458155 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:54.459019 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:54.459305 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:54.459604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:54.461493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:54.530154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:54.531642 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:54.542735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1326/2682] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:46:55.720604 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3f35e-f922-464a-9e3b-5638100716ee None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:55.721062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3f35e-f922-464a-9e3b-5638100716ee None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:55.721237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3f35e-f922-464a-9e3b-5638100716ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:55.760473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-7af3f35e-f922-464a-9e3b-5638100716ee None None] MFA Rules not processed for user `63c0c061be824eae8d217d7ea32dcd92`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:46:55.766850 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-7af3f35e-f922-464a-9e3b-5638100716ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:55.852502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1357/2683] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:46:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:46:58.511552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:46:58.513204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:58.611478 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:46:58.679930 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96810f9e-cd1e-43d0-8649-09d22b6b83e8 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:46:58.706550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:46:58.707841 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:46:58.708300 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:46:58.708591 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:46:58.715039 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:46:58.730061 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:46:58.731675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:46:58.746674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1327/2684] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:46:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:47:00.938267 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f451e-3b39-42ab-8408-aeda8dc9f654 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:00.938933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f451e-3b39-42ab-8408-aeda8dc9f654 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:00.938933 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f451e-3b39-42ab-8408-aeda8dc9f654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:00.992693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-381f451e-3b39-42ab-8408-aeda8dc9f654 None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:00.997287 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-381f451e-3b39-42ab-8408-aeda8dc9f654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:01.112899 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1358/2685] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:47:01.133114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:01.134765 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:01.267770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c613a9c-3300-4995-becb-950cfc0998dd service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:01.275047 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393d49-1788-4286-91bb-b3b84c67064b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:01.275717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393d49-1788-4286-91bb-b3b84c67064b service swift] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:01.275988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393d49-1788-4286-91bb-b3b84c67064b service swift] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:01.276400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393d49-1788-4286-91bb-b3b84c67064b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:01.291531 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9393d49-1788-4286-91bb-b3b84c67064b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:01.294980 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9393d49-1788-4286-91bb-b3b84c67064b service swift] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:01.314579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1328/2686] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:47:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:47:02.816063 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f451e-3b39-42ab-8408-aeda8dc9f654 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:02.819074 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-381f451e-3b39-42ab-8408-aeda8dc9f654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:02.832139 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381f451e-3b39-42ab-8408-aeda8dc9f654 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:02.845978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:02.846432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:02.846690 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:02.847169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:02.850363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:02.962270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:02.979033 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:02.980693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:02.995569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1359/2687] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:47:11.306083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=48 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:12.989685 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:13.370743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393d49-1788-4286-91bb-b3b84c67064b service swift] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:13.385363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393d49-1788-4286-91bb-b3b84c67064b service swift] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:13.390660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:13.391264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:13.391567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:13.391884 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:13.393716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:13.407071 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:13.408797 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:13.420028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1329/2688] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:47:13.529583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:13.531019 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:13.544435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47bd02-f2d3-4b07-a637-a6a910385747 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:13.550602 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:13.551252 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:13.551596 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:13.551869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] PATH_INFO: `/v3/users/2730741d853a4cfcab1bbc990cf42611` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:13.553104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2730741d853a4cfcab1bbc990cf42611)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:13.555047 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:13.769837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2730741d853a4cfcab1bbc990cf42611'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:13.771169 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2730741d853a4cfcab1bbc990cf42611'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:13.778750 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2730741d853a4cfcab1bbc990cf42611'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:13.860495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2730741d853a4cfcab1bbc990cf42611'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:13.861151 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2730741d853a4cfcab1bbc990cf42611'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:13.881728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1360/2689] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:47:13 2022] DELETE /identity/v3/users/2730741d853a4cfcab1bbc990cf42611 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:14.148271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:14.161828 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f86db0-3c16-470f-b2d1-ef893c698bc9 service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:14.222092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:14.222733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:14.223018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:14.223713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] PATH_INFO: `/v3/projects/6bcafe839dec4fcc95a4c720463483ba` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:14.225170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bcafe839dec4fcc95a4c720463483ba)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:14.227085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:14.358238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bcafe839dec4fcc95a4c720463483ba', 'request_id': 'req-ab428971-0663-4327-85f6-1a8d991f1f52'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.365859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bcafe839dec4fcc95a4c720463483ba', 'request_id': 'req-ab428971-0663-4327-85f6-1a8d991f1f52'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.372060 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bcafe839dec4fcc95a4c720463483ba', 'request_id': 'req-ab428971-0663-4327-85f6-1a8d991f1f52'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.397625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:14.399279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:14.413693 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422848f-b4af-43c3-a3aa-98348e4ff48a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:14.479720 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:14.481355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:14.481355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:14.481355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] PATH_INFO: `/v3/users/38f8522facb34b9b9660136e4dd214f2` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:14.482914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f8522facb34b9b9660136e4dd214f2)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:14.484996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:14.485128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] The token cache is being invalidate because project 6bcafe839dec4fcc95a4c720463483ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:47:14.485401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.486056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.487567 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1330/2690] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:47:14 2022] DELETE /identity/v3/projects/6bcafe839dec4fcc95a4c720463483ba => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:14.492982 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:14.494674 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:14.509070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab428971-0663-4327-85f6-1a8d991f1f52 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:14.514380 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:14.514859 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:14.515353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:14.515632 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] PATH_INFO: `/v3/users/cc286bfe31e04fc89d411e1bdcc54a84` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:14.516822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc286bfe31e04fc89d411e1bdcc54a84)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:14.518766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:14.818283 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f8522facb34b9b9660136e4dd214f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.818997 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f8522facb34b9b9660136e4dd214f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.826080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f8522facb34b9b9660136e4dd214f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.852752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc286bfe31e04fc89d411e1bdcc54a84'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.853440 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc286bfe31e04fc89d411e1bdcc54a84'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.860552 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc286bfe31e04fc89d411e1bdcc54a84'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.867247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc286bfe31e04fc89d411e1bdcc54a84'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.924305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f8522facb34b9b9660136e4dd214f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.924951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-72cd7a68-23f1-48cc-a638-f7cb61f32e52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f8522facb34b9b9660136e4dd214f2'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:14.935605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:14.944094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1361/2691] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:47:14 2022] DELETE /identity/v3/users/38f8522facb34b9b9660136e4dd214f2 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:47:14.949295 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42010a01-dcee-401c-81c1-19f3550c6e3d None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:14.949721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42010a01-dcee-401c-81c1-19f3550c6e3d None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:14.950135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42010a01-dcee-401c-81c1-19f3550c6e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:14.974301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-42010a01-dcee-401c-81c1-19f3550c6e3d None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:14.975575 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-42010a01-dcee-401c-81c1-19f3550c6e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:15.032127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc286bfe31e04fc89d411e1bdcc54a84'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:15.034924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1362/2692] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:47:15.041965 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:15.046034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42010a01-dcee-401c-81c1-19f3550c6e3d None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:15.047643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-42010a01-dcee-401c-81c1-19f3550c6e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:15.053960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1331/2693] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:47:14 2022] DELETE /identity/v3/users/cc286bfe31e04fc89d411e1bdcc54a84 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:15.120001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42010a01-dcee-401c-81c1-19f3550c6e3d None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:15.187212 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:15.188615 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:15.188902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:15.189177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:15.266133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:15.267635 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:15.273185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:15.281726 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1363/2694] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:47:15.338072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:15.339951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:15.362847 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd0a27-32ca-469c-8468-6567a0448e04 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:15.373701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:15.373701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:15.373701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:15.373701 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] PATH_INFO: `/v3/projects/375bbf2a674b43e2927d3b4ab3f6f886` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:15.378649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375bbf2a674b43e2927d3b4ab3f6f886)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:15.382024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:15.498295 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375bbf2a674b43e2927d3b4ab3f6f886', 'request_id': 'req-895dc212-2ad3-44d5-9c0f-76d723c91630'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:15.506717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375bbf2a674b43e2927d3b4ab3f6f886', 'request_id': 'req-895dc212-2ad3-44d5-9c0f-76d723c91630'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:15.514304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375bbf2a674b43e2927d3b4ab3f6f886', 'request_id': 'req-895dc212-2ad3-44d5-9c0f-76d723c91630'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:15.587109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] The token cache is being invalidate because project 375bbf2a674b43e2927d3b4ab3f6f886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:47:15.587483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:15.588236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-895dc212-2ad3-44d5-9c0f-76d723c91630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:15.604259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1332/2695] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:47:15 2022] DELETE /identity/v3/projects/375bbf2a674b43e2927d3b4ab3f6f886 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:15.920599 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:15.922126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:15.936552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57146a-249e-4a8c-b84f-7ff90473991d service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:16.004036 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:16.005334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.005628 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.005911 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] PATH_INFO: `/v3/projects/face821ff8c7474ebdc02058b43dc575` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.007109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=face821ff8c7474ebdc02058b43dc575)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:16.009093 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:16.108354 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'face821ff8c7474ebdc02058b43dc575', 'request_id': 'req-6465f8b2-4228-4d68-b759-d49aa3f1a329'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:16.115245 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'face821ff8c7474ebdc02058b43dc575', 'request_id': 'req-6465f8b2-4228-4d68-b759-d49aa3f1a329'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:16.198376 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'face821ff8c7474ebdc02058b43dc575', 'request_id': 'req-6465f8b2-4228-4d68-b759-d49aa3f1a329'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:16.211260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] The token cache is being invalidate because project face821ff8c7474ebdc02058b43dc575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:47:16.211633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:16.212229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-6465f8b2-4228-4d68-b759-d49aa3f1a329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:16.213752 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1364/2696] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:47:15 2022] DELETE /identity/v3/projects/face821ff8c7474ebdc02058b43dc575 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:16.219609 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3809e0-bbf5-48b2-bf73-2cd71e7f0300 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.219902 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3809e0-bbf5-48b2-bf73-2cd71e7f0300 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.220203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3809e0-bbf5-48b2-bf73-2cd71e7f0300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.242193 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-2d3809e0-bbf5-48b2-bf73-2cd71e7f0300 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:16.243432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-2d3809e0-bbf5-48b2-bf73-2cd71e7f0300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:16.306387 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1333/2697] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:47:16.314301 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71d5c3-59ff-497d-a052-a699172d420f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.314583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71d5c3-59ff-497d-a052-a699172d420f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.314861 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71d5c3-59ff-497d-a052-a699172d420f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.337699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ec71d5c3-59ff-497d-a052-a699172d420f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:16.338949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ec71d5c3-59ff-497d-a052-a699172d420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:16.347079 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1365/2698] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:47:16.352098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3809e0-bbf5-48b2-bf73-2cd71e7f0300 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:16.362177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3809e0-bbf5-48b2-bf73-2cd71e7f0300 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:16.366889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:16.367490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.367850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.368128 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.368819 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:47:16.369120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:16.370729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:16.383242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1334/2699] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:47:16.387895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71d5c3-59ff-497d-a052-a699172d420f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:16.398870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71d5c3-59ff-497d-a052-a699172d420f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:16.403709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f1c00-c3fa-42c8-bdab-611fa921a2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:16.404355 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f1c00-c3fa-42c8-bdab-611fa921a2b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.404647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f1c00-c3fa-42c8-bdab-611fa921a2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.404922 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f1c00-c3fa-42c8-bdab-611fa921a2b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.405559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f1c00-c3fa-42c8-bdab-611fa921a2b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:16.407476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f1c00-c3fa-42c8-bdab-611fa921a2b7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:16.514152 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1366/2700] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:47:16 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:47:16.518574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:16.529443 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cef553-651b-429c-8eb7-bc49985c1456 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:16.589510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c204-a2c5-42c9-a212-1eea382ce8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:16.590115 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c204-a2c5-42c9-a212-1eea382ce8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.590396 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c204-a2c5-42c9-a212-1eea382ce8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.590671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12c204-a2c5-42c9-a212-1eea382ce8f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.591319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12c204-a2c5-42c9-a212-1eea382ce8f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:16.593271 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12c204-a2c5-42c9-a212-1eea382ce8f1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:16.602026 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:16.707471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1335/2701] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:47:16 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:47:16.712642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f1c00-c3fa-42c8-bdab-611fa921a2b7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:16.726756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f1c00-c3fa-42c8-bdab-611fa921a2b7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:16.737352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19df7c-5e45-46b2-ae73-1d2f875a74c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:16.737958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19df7c-5e45-46b2-ae73-1d2f875a74c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.738238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19df7c-5e45-46b2-ae73-1d2f875a74c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.751489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19df7c-5e45-46b2-ae73-1d2f875a74c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.751489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19df7c-5e45-46b2-ae73-1d2f875a74c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:16.751489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19df7c-5e45-46b2-ae73-1d2f875a74c1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:16.752382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1367/2702] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:47:16.756785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c204-a2c5-42c9-a212-1eea382ce8f1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:16.767083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12c204-a2c5-42c9-a212-1eea382ce8f1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:16.773059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f1bb4-76c8-4272-a427-20361a89c07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:16.773676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f1bb4-76c8-4272-a427-20361a89c07d admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.774024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f1bb4-76c8-4272-a427-20361a89c07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.774307 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f1bb4-76c8-4272-a427-20361a89c07d admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.775002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f1bb4-76c8-4272-a427-20361a89c07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:16.776679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f1bb4-76c8-4272-a427-20361a89c07d admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:16.788460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1336/2703] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:47:16.792672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19df7c-5e45-46b2-ae73-1d2f875a74c1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:16.803159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19df7c-5e45-46b2-ae73-1d2f875a74c1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:16.807981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:16.808662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.808943 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.809217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] PATH_INFO: `/v3/projects/ad1bfd3bb3d847b999593480bc67d1d6/users/b138a77a6a3b431382efb9d0c4910607/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.835264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1bfd3bb3d847b999593480bc67d1d6, user_id=b138a77a6a3b431382efb9d0c4910607, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:16.838204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:16.921836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1368/2704] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:47:16 2022] PUT /identity/v3/projects/ad1bfd3bb3d847b999593480bc67d1d6/users/b138a77a6a3b431382efb9d0c4910607/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:16.927559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021bb11-e2b4-4f5c-b828-2b3d53b0a204 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:16.927900 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021bb11-e2b4-4f5c-b828-2b3d53b0a204 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:16.928199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021bb11-e2b4-4f5c-b828-2b3d53b0a204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:16.958988 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-c021bb11-e2b4-4f5c-b828-2b3d53b0a204 None None] MFA Rules not processed for user `b138a77a6a3b431382efb9d0c4910607`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:16.960206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c021bb11-e2b4-4f5c-b828-2b3d53b0a204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:17.058334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1337/2705] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:47:17.071931 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:17.073730 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:17.105277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:17.146100 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07c0f3-f691-4bc6-80e2-c60cbdc7e96d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:17.162535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbe581-bbfd-448c-a598-7a39b433979b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:17.162535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbe581-bbfd-448c-a598-7a39b433979b service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:17.162535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbe581-bbfd-448c-a598-7a39b433979b service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:17.162535 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbe581-bbfd-448c-a598-7a39b433979b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:17.162975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbe581-bbfd-448c-a598-7a39b433979b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:17.163484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbe581-bbfd-448c-a598-7a39b433979b service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:17.174206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1369/2706] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:47:27.055083 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:27.364511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=15 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:30.048112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021bb11-e2b4-4f5c-b828-2b3d53b0a204 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:30.113920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021bb11-e2b4-4f5c-b828-2b3d53b0a204 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:30.118513 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:30.119132 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:30.119409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:30.119677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] PATH_INFO: `/v3/users/b138a77a6a3b431382efb9d0c4910607` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:30.120826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b138a77a6a3b431382efb9d0c4910607)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:30.127453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:30.349851 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b138a77a6a3b431382efb9d0c4910607'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:30.350566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b138a77a6a3b431382efb9d0c4910607'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:30.357858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b138a77a6a3b431382efb9d0c4910607'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:30.358510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b138a77a6a3b431382efb9d0c4910607'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:30.495508 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b138a77a6a3b431382efb9d0c4910607'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:30.520471 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1338/2707] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:47:30 2022] DELETE /identity/v3/users/b138a77a6a3b431382efb9d0c4910607 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:30.522898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1839529-a802-4707-9186-4bc7e3358b6e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:30.523405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1839529-a802-4707-9186-4bc7e3358b6e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:30.523495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1839529-a802-4707-9186-4bc7e3358b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:30.548700 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b1839529-a802-4707-9186-4bc7e3358b6e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:30.549924 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b1839529-a802-4707-9186-4bc7e3358b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:30.612331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1370/2708] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:47:30.623297 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:30.624996 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:30.687400 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6641d8-6fad-4f2d-9a60-bd4741579776 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:30.691428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc763f-54f5-47e4-88ec-214ee5d072cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:30.692034 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc763f-54f5-47e4-88ec-214ee5d072cc service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:30.692348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc763f-54f5-47e4-88ec-214ee5d072cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:30.693332 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc763f-54f5-47e4-88ec-214ee5d072cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:30.703586 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc763f-54f5-47e4-88ec-214ee5d072cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:30.705122 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc763f-54f5-47e4-88ec-214ee5d072cc service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:30.714770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1339/2709] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:47:31.243238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1839529-a802-4707-9186-4bc7e3358b6e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:31.243238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b1839529-a802-4707-9186-4bc7e3358b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:31.253867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1839529-a802-4707-9186-4bc7e3358b6e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:31.260171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:31.260907 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:31.261189 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:31.261464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] PATH_INFO: `/v3/projects/ad1bfd3bb3d847b999593480bc67d1d6` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:31.262827 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1bfd3bb3d847b999593480bc67d1d6)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:31.264903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:31.393589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1bfd3bb3d847b999593480bc67d1d6', 'request_id': 'req-b6209e8d-7e47-4751-9239-b76fcd07cd3f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:31.399866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1bfd3bb3d847b999593480bc67d1d6', 'request_id': 'req-b6209e8d-7e47-4751-9239-b76fcd07cd3f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:31.460760 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1bfd3bb3d847b999593480bc67d1d6', 'request_id': 'req-b6209e8d-7e47-4751-9239-b76fcd07cd3f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:31.472371 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] The token cache is being invalidate because project ad1bfd3bb3d847b999593480bc67d1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:47:31.472657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:31.473272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-b6209e8d-7e47-4751-9239-b76fcd07cd3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:47:31.475835 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1371/2710] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:47:31 2022] DELETE /identity/v3/projects/ad1bfd3bb3d847b999593480bc67d1d6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:31.481222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bdf11-da70-4f92-9a63-487b3ca90adb None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:31.481845 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bdf11-da70-4f92-9a63-487b3ca90adb None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:31.481956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bdf11-da70-4f92-9a63-487b3ca90adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:31.502923 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-888bdf11-da70-4f92-9a63-487b3ca90adb None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:31.504151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-888bdf11-da70-4f92-9a63-487b3ca90adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:31.525181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:31.576222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1340/2711] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:47:31.581656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b0c4-9da7-4f90-974b-21a61aec45fd None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:31.582133 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b0c4-9da7-4f90-974b-21a61aec45fd None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:31.582233 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b0c4-9da7-4f90-974b-21a61aec45fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:31.621224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-e6d6b0c4-9da7-4f90-974b-21a61aec45fd None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:31.621224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-e6d6b0c4-9da7-4f90-974b-21a61aec45fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:31.627207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1372/2712] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:47:31.632722 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bdf11-da70-4f92-9a63-487b3ca90adb None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:31.643353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bdf11-da70-4f92-9a63-487b3ca90adb None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:31.648724 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:31.649451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:31.649735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:31.650025 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:31.650655 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:47:31.650956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:31.652576 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:31.664790 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1341/2713] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:47:31.673382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b0c4-9da7-4f90-974b-21a61aec45fd None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:31.678788 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b0c4-9da7-4f90-974b-21a61aec45fd None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:31.683518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64949456-3bc8-4d85-9e20-e812d80b8aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:31.684115 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64949456-3bc8-4d85-9e20-e812d80b8aaf admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:31.684430 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64949456-3bc8-4d85-9e20-e812d80b8aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:31.684702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64949456-3bc8-4d85-9e20-e812d80b8aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:31.685338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64949456-3bc8-4d85-9e20-e812d80b8aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:31.687217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64949456-3bc8-4d85-9e20-e812d80b8aaf admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:31.862504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1373/2714] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:47:31 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:47:31.866750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:31.876901 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa3929-3f84-4337-a843-d2aa9a05fc4a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:31.935566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4fba8-ad89-46cc-a860-fcc90e452eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:31.936199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4fba8-ad89-46cc-a860-fcc90e452eae admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:31.936489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4fba8-ad89-46cc-a860-fcc90e452eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:31.936767 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4fba8-ad89-46cc-a860-fcc90e452eae admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:31.937439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4fba8-ad89-46cc-a860-fcc90e452eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:31.939463 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4fba8-ad89-46cc-a860-fcc90e452eae admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:32.076217 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1342/2715] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:47:31 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:47:32.077529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64949456-3bc8-4d85-9e20-e812d80b8aaf admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:32.090170 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64949456-3bc8-4d85-9e20-e812d80b8aaf admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:32.095336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475fb86-49f7-4ee1-9f97-3f7dd251c16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:32.095967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475fb86-49f7-4ee1-9f97-3f7dd251c16c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:32.096278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475fb86-49f7-4ee1-9f97-3f7dd251c16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:32.096557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475fb86-49f7-4ee1-9f97-3f7dd251c16c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:32.097493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475fb86-49f7-4ee1-9f97-3f7dd251c16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:32.099155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475fb86-49f7-4ee1-9f97-3f7dd251c16c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:32.110475 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1374/2716] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:47:32.114726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4fba8-ad89-46cc-a860-fcc90e452eae admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:32.126934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4fba8-ad89-46cc-a860-fcc90e452eae admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:32.132451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b3507-205e-40e4-8b33-7106d0c36fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:32.133063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b3507-205e-40e4-8b33-7106d0c36fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:32.133376 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b3507-205e-40e4-8b33-7106d0c36fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:32.133670 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b3507-205e-40e4-8b33-7106d0c36fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:32.134314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b3507-205e-40e4-8b33-7106d0c36fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:32.135917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b3507-205e-40e4-8b33-7106d0c36fcf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:32.151987 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1343/2717] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:47:32.153154 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475fb86-49f7-4ee1-9f97-3f7dd251c16c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:32.164322 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475fb86-49f7-4ee1-9f97-3f7dd251c16c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:32.170631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:32.171304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:32.171591 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:32.174390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] PATH_INFO: `/v3/projects/1a7c5f7e89a74cab86be21c2bb4819ab/users/e9ee68b458234fe482b5c2e777b71967/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:32.189383 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a7c5f7e89a74cab86be21c2bb4819ab, user_id=e9ee68b458234fe482b5c2e777b71967, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:32.192580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:32.312794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1375/2718] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:47:32 2022] PUT /identity/v3/projects/1a7c5f7e89a74cab86be21c2bb4819ab/users/e9ee68b458234fe482b5c2e777b71967/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:47:32.318198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9de1-4d28-4f6b-bff4-3c5465936172 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:32.318617 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9de1-4d28-4f6b-bff4-3c5465936172 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:32.318903 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e9de1-4d28-4f6b-bff4-3c5465936172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:32.340004 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b51e9de1-4d28-4f6b-bff4-3c5465936172 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:32.341265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b51e9de1-4d28-4f6b-bff4-3c5465936172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:32.404241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1344/2719] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:47:32.414223 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:32.416089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:32.439027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:32.476239 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ff135-bcb3-42b5-9da2-47ebcf71efd0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:32.479944 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:32.480578 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:32.480851 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:32.481117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:32.490532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:32.491949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:32.501600 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1376/2720] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:47:34.068206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e9dd-8bdb-4550-9dce-126c4f8e8e6d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:34.069869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e9dd-8bdb-4550-9dce-126c4f8e8e6d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:34.070144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e9dd-8bdb-4550-9dce-126c4f8e8e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:34.103251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d0a6e9dd-8bdb-4550-9dce-126c4f8e8e6d None None] MFA Rules not processed for user `e9ee68b458234fe482b5c2e777b71967`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:34.104500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d0a6e9dd-8bdb-4550-9dce-126c4f8e8e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:34.121711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:34.171530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1345/2721] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:47:34.182891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:34.184396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:34.260977 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666a33f-0c8a-425e-b30c-ec0cb74b3960 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:34.264747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0df54-4044-4c28-a415-b7d5229891c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:34.265483 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0df54-4044-4c28-a415-b7d5229891c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:34.265766 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0df54-4044-4c28-a415-b7d5229891c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:34.266180 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0df54-4044-4c28-a415-b7d5229891c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:34.274829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0df54-4044-4c28-a415-b7d5229891c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:34.276375 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d0df54-4044-4c28-a415-b7d5229891c3 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:34.285776 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1377/2722] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:47:36.060976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f57bd-f08c-4f39-8d47-0a68303a9ea7 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:36.061506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f57bd-f08c-4f39-8d47-0a68303a9ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:36.061506 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f57bd-f08c-4f39-8d47-0a68303a9ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:36.081753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3b3f57bd-f08c-4f39-8d47-0a68303a9ea7 None None] MFA Rules not processed for user `e9ee68b458234fe482b5c2e777b71967`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:36.083037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3b3f57bd-f08c-4f39-8d47-0a68303a9ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:36.091418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1346/2723] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:47:40.901791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0df54-4044-4c28-a415-b7d5229891c3 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:40.911088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0df54-4044-4c28-a415-b7d5229891c3 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:40.915204 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150a9f1-f6e4-4e5f-9fa9-a7bbb4747e29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:40.915830 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150a9f1-f6e4-4e5f-9fa9-a7bbb4747e29 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:40.916109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150a9f1-f6e4-4e5f-9fa9-a7bbb4747e29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:40.916414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150a9f1-f6e4-4e5f-9fa9-a7bbb4747e29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:40.918215 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1150a9f1-f6e4-4e5f-9fa9-a7bbb4747e29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:40.978631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150a9f1-f6e4-4e5f-9fa9-a7bbb4747e29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:40.980058 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150a9f1-f6e4-4e5f-9fa9-a7bbb4747e29 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:40.989246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1378/2724] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:47:41.593975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f57bd-f08c-4f39-8d47-0a68303a9ea7 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:47:41.595431 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3b3f57bd-f08c-4f39-8d47-0a68303a9ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:41.608450 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f57bd-f08c-4f39-8d47-0a68303a9ea7 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:47:41.612252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:47:41.612875 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:41.613180 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:41.613454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:41.615213 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:41.686523 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:47:41.688134 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:47:41.699823 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1347/2725] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:47:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:47:44.336865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00be68-0c60-4b8f-9833-e0a14462bad4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:47:44.337447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00be68-0c60-4b8f-9833-e0a14462bad4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:47:44.337842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00be68-0c60-4b8f-9833-e0a14462bad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:47:44.372880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-dc00be68-0c60-4b8f-9833-e0a14462bad4 None None] MFA Rules not processed for user `98b30f0436864263b2829921f212f8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:47:44.375755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dc00be68-0c60-4b8f-9833-e0a14462bad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:47:44.386951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1379/2726] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:47:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:47:51.695347 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=52 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:47:54.363532 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=74 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:48:12.209151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:48:12.210692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:48:12.222404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed71c2-f58d-4d5d-9d41-2d4f455d1641 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:48:12.226565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449420d-5b3e-44f1-a863-ce409ef009a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:48:12.227163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449420d-5b3e-44f1-a863-ce409ef009a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:48:12.227475 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449420d-5b3e-44f1-a863-ce409ef009a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:48:12.227750 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449420d-5b3e-44f1-a863-ce409ef009a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:48:12.229486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5449420d-5b3e-44f1-a863-ce409ef009a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:48:12.302020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449420d-5b3e-44f1-a863-ce409ef009a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:48:12.304209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449420d-5b3e-44f1-a863-ce409ef009a7 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:48:12.316339 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1348/2727] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:48:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:48:13.869088 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00be68-0c60-4b8f-9833-e0a14462bad4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:48:13.879254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00be68-0c60-4b8f-9833-e0a14462bad4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:48:13.884349 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:48:13.884522 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:48:13.884803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:48:13.885096 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:48:13.886824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:48:13.902390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:48:13.903829 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:48:13.915229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1380/2728] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:48:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:48:22.311394 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=34 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:48:23.910107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:48:58.112912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefab0e-af89-4484-aefb-e9552e664e93 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:48:58.114308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefab0e-af89-4484-aefb-e9552e664e93 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:48:58.114308 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefab0e-af89-4484-aefb-e9552e664e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:48:58.134263 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4aefab0e-af89-4484-aefb-e9552e664e93 None None] MFA Rules not processed for user `e9ee68b458234fe482b5c2e777b71967`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:48:58.135499 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4aefab0e-af89-4484-aefb-e9552e664e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:48:58.142718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1349/2729] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:48:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:49:02.024067 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:49:02.033266 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848eefe-37f9-47d3-93d0-a35bdc135c1a service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:49:02.037897 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:49:02.038436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:02.038721 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:02.038999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:02.040842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:02.050958 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:49:02.052521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:49:02.061187 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1381/2730] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:49:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:49:08.128054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:49:09.801958 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefab0e-af89-4484-aefb-e9552e664e93 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:49:09.805429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4aefab0e-af89-4484-aefb-e9552e664e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:09.853578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefab0e-af89-4484-aefb-e9552e664e93 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:49:09.863158 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f27e5-eec1-42d1-b369-8dc837736e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:49:09.864765 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f27e5-eec1-42d1-b369-8dc837736e51 service nova] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:09.865969 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f27e5-eec1-42d1-b369-8dc837736e51 service nova] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:09.866840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f27e5-eec1-42d1-b369-8dc837736e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:09.870928 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0f9f27e5-eec1-42d1-b369-8dc837736e51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:09.895476 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f27e5-eec1-42d1-b369-8dc837736e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:49:09.899568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f27e5-eec1-42d1-b369-8dc837736e51 service nova] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:49:09.929015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1350/2731] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:49:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:49:12.057060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:49:19.012744 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:49:19.014662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:19.032120 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6326814-a78b-411a-985b-693b220e4128 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:49:19.039437 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:49:19.043062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:19.043062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:19.043062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:19.043062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:19.056866 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:49:19.079562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:49:19.079691 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1382/2732] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:49:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:49:19.909124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=6 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:49:20.933700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fb874-6300-420a-87f2-1deede853e8e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:20.937743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fb874-6300-420a-87f2-1deede853e8e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:20.937743 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fb874-6300-420a-87f2-1deede853e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:20.995268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d46fb874-6300-420a-87f2-1deede853e8e None None] MFA Rules not processed for user `b56fb09428cb4ab2b49ca199058a5a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:49:20.996528 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d46fb874-6300-420a-87f2-1deede853e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:21.060892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1351/2733] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 13:49:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:49:21.105075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:49:21.106516 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:21.206224 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b5a09-3e5c-48ce-99d3-9f8dc0163ec8 service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:49:21.212362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea626e8-726a-40e1-8122-6298e5d5121f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd886328304543828dca35df3a0b7411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:49:21.212971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea626e8-726a-40e1-8122-6298e5d5121f service swift] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:21.213254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea626e8-726a-40e1-8122-6298e5d5121f service swift] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:21.213531 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea626e8-726a-40e1-8122-6298e5d5121f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:21.222121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea626e8-726a-40e1-8122-6298e5d5121f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:49:21.223749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea626e8-726a-40e1-8122-6298e5d5121f service swift] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:49:21.232742 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1383/2734] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 13:49:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 13:49:30.284350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fb874-6300-420a-87f2-1deede853e8e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:49:30.284350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d46fb874-6300-420a-87f2-1deede853e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:30.301051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fb874-6300-420a-87f2-1deede853e8e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:49:30.307253 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:49:30.307878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:30.308166 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:30.308785 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] PATH_INFO: `/v3/users/63c0c061be824eae8d217d7ea32dcd92` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:30.310048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c0c061be824eae8d217d7ea32dcd92)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:49:30.312002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:49:30.535228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c0c061be824eae8d217d7ea32dcd92'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:30.535889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c0c061be824eae8d217d7ea32dcd92'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:30.543829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c0c061be824eae8d217d7ea32dcd92'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:30.544501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c0c061be824eae8d217d7ea32dcd92'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:30.619018 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c0c061be824eae8d217d7ea32dcd92'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:30.637697 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1352/2735] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:49:30 2022] DELETE /identity/v3/users/63c0c061be824eae8d217d7ea32dcd92 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:49:30.888996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea626e8-726a-40e1-8122-6298e5d5121f service swift] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:49:30.902521 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea626e8-726a-40e1-8122-6298e5d5121f service swift] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:49:30.970241 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:49:30.970885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:30.971173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:30.971454 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] PATH_INFO: `/v3/projects/6f61a8f100cf4d39928e4d596741e1bb` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:30.972694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f61a8f100cf4d39928e4d596741e1bb)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:49:30.976167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:49:31.101838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f61a8f100cf4d39928e4d596741e1bb', 'request_id': 'req-a7da16b4-a623-48d5-90ff-790d423a28a6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.107451 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f61a8f100cf4d39928e4d596741e1bb', 'request_id': 'req-a7da16b4-a623-48d5-90ff-790d423a28a6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.109165 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:49:31.213106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f61a8f100cf4d39928e4d596741e1bb', 'request_id': 'req-a7da16b4-a623-48d5-90ff-790d423a28a6'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.224898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] The token cache is being invalidate because project 6f61a8f100cf4d39928e4d596741e1bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:49:31.225182 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.225859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.227455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1384/2736] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:49:30 2022] DELETE /identity/v3/projects/6f61a8f100cf4d39928e4d596741e1bb => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:49:31.231520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:49:31.233043 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:31.247012 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52ccca-088a-45fd-b4d7-a9fe15d33d6f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:49:31.289945 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:49:31.304946 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:49:31.305593 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:31.305873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:31.306149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] PATH_INFO: `/v3/users/4c8657afe9604c20a4c92f8195adcb1c` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:31.307302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c8657afe9604c20a4c92f8195adcb1c)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:49:31.309363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:49:31.492099 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c8657afe9604c20a4c92f8195adcb1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.492794 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c8657afe9604c20a4c92f8195adcb1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.499994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c8657afe9604c20a4c92f8195adcb1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.500707 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c8657afe9604c20a4c92f8195adcb1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.590370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7f6ff2af-bd0e-42c7-ab02-538d230eb756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c8657afe9604c20a4c92f8195adcb1c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:31.609162 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1353/2737] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:49:31 2022] DELETE /identity/v3/users/4c8657afe9604c20a4c92f8195adcb1c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:49:31.889898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:49:31.891611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:49:31.913122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da16b4-a623-48d5-90ff-790d423a28a6 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:49:31.993748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:49:31.994819 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:49:31.995136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:49:31.995413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] PATH_INFO: `/v3/projects/baa4fb911d194621bcf35c84c846ea9c` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:49:31.996765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa4fb911d194621bcf35c84c846ea9c)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:49:31.998999 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:49:32.119860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa4fb911d194621bcf35c84c846ea9c', 'request_id': 'req-20c4bd16-7896-4397-b447-66d7c322211e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:32.125860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa4fb911d194621bcf35c84c846ea9c', 'request_id': 'req-20c4bd16-7896-4397-b447-66d7c322211e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:32.288075 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa4fb911d194621bcf35c84c846ea9c', 'request_id': 'req-20c4bd16-7896-4397-b447-66d7c322211e'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:32.301146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] The token cache is being invalidate because project baa4fb911d194621bcf35c84c846ea9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:49:32.301442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:32.302102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:49:32.304643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1385/2738] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:49:31 2022] DELETE /identity/v3/projects/baa4fb911d194621bcf35c84c846ea9c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:49:41.605482 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:49:42.297715 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=55 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:50:08.814735 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7f58-32e5-4dbd-949a-1033bdd98bb6 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:08.815311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7f58-32e5-4dbd-949a-1033bdd98bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:08.815311 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7f58-32e5-4dbd-949a-1033bdd98bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:08.835610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-12af7f58-32e5-4dbd-949a-1033bdd98bb6 None None] MFA Rules not processed for user `33835c5512df4c80a9b7cb9712a5904f`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:50:08.837046 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-12af7f58-32e5-4dbd-949a-1033bdd98bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:08.895911 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1354/2739] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:50:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:50:09.219432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:09.221117 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:09.286171 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4bd16-7896-4397-b447-66d7c322211e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:09.289784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:09.290389 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:09.290666 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:09.290937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:09.292963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:09.303439 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:09.304952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:09.314903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1386/2740] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:50:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:50:09.724238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7f58-32e5-4dbd-949a-1033bdd98bb6 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:09.726127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-12af7f58-32e5-4dbd-949a-1033bdd98bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:09.814298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7f58-32e5-4dbd-949a-1033bdd98bb6 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:09.819572 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:09.820197 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:09.825510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:09.825510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:09.825510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:09.834764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:09.836386 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:09.847445 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1355/2741] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:50:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:50:10.295411 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:10.297300 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:10.368920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2b015-0fe0-4a39-a56a-a4de1d29507d service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:10.373748 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:10.374384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:10.374662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:10.374942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] PATH_INFO: `/v3/users/e9ee68b458234fe482b5c2e777b71967` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:10.376340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ee68b458234fe482b5c2e777b71967)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:10.378272 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:10.616443 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ee68b458234fe482b5c2e777b71967'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:10.617124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ee68b458234fe482b5c2e777b71967'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:10.625260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ee68b458234fe482b5c2e777b71967'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:10.687646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ee68b458234fe482b5c2e777b71967'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:10.688381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ee68b458234fe482b5c2e777b71967'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:10.707978 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1387/2742] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:50:10 2022] DELETE /identity/v3/users/e9ee68b458234fe482b5c2e777b71967 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:50:10.986042 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:10.997835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17806dbd-822a-4a44-a9ce-a523254367fc service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:11.056574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:11.057737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:11.058014 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:11.058289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] PATH_INFO: `/v3/projects/1a7c5f7e89a74cab86be21c2bb4819ab` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:11.059418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a7c5f7e89a74cab86be21c2bb4819ab)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:11.061390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:11.183320 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:50:11.189052 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a7c5f7e89a74cab86be21c2bb4819ab', 'request_id': 'req-0b188f1b-26c5-459d-b745-ce00a8f9374c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:11.195439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a7c5f7e89a74cab86be21c2bb4819ab', 'request_id': 'req-0b188f1b-26c5-459d-b745-ce00a8f9374c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:11.203871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a7c5f7e89a74cab86be21c2bb4819ab', 'request_id': 'req-0b188f1b-26c5-459d-b745-ce00a8f9374c'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:11.311729 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] The token cache is being invalidate because project 1a7c5f7e89a74cab86be21c2bb4819ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:50:11.312330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:11.313173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:11.314864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1356/2743] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:50:10 2022] DELETE /identity/v3/projects/1a7c5f7e89a74cab86be21c2bb4819ab => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:50:19.462792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:19.465809 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:19.502369 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:50:19.555050 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130eabc-4d45-466f-9e34-5b0859e1d5c8 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:19.629891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40561a306123461d92bc669be3962db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:19.630518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:19.630805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:19.631080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:19.633011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:19.646966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:19.649433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:19.661612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1388/2744] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:50:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:50:19.915135 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:19.917867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:19.983442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188f1b-26c5-459d-b745-ce00a8f9374c admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:19.987079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:19.987677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:19.987954 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:19.988245 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:19.989990 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:20.057858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:20.059444 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:20.069533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1357/2745] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:50:28.322937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:28.324605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:28.340761 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52e2b7-cfbd-4115-a880-4de8513b4b4b service nova] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:28.345588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:28.346238 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:28.346517 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:28.346793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] PATH_INFO: `/v3/users/98b30f0436864263b2829921f212f8ce` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:28.347942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b30f0436864263b2829921f212f8ce)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:28.349914 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:28.604499 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b30f0436864263b2829921f212f8ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:28.605259 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b30f0436864263b2829921f212f8ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:28.612699 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b30f0436864263b2829921f212f8ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:28.680970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b30f0436864263b2829921f212f8ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:28.681588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b30f0436864263b2829921f212f8ce'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:28.700626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1389/2746] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:50:28 2022] DELETE /identity/v3/users/98b30f0436864263b2829921f212f8ce => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:50:29.003179 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:29.014693 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f501f11-f787-433a-94b3-c40d623b967a service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:29.081041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:29.081717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:29.082002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:29.082281 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] PATH_INFO: `/v3/projects/6842f55d3dcc42289b6bfe02ed020dc4` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:29.083418 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6842f55d3dcc42289b6bfe02ed020dc4)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:29.085366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:29.102792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:50:29.219811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6842f55d3dcc42289b6bfe02ed020dc4', 'request_id': 'req-7afdcf4c-3381-464a-bc42-a91b2c291e6f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.225223 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6842f55d3dcc42289b6bfe02ed020dc4', 'request_id': 'req-7afdcf4c-3381-464a-bc42-a91b2c291e6f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.231997 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6842f55d3dcc42289b6bfe02ed020dc4', 'request_id': 'req-7afdcf4c-3381-464a-bc42-a91b2c291e6f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.332639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] The token cache is being invalidate because project 6842f55d3dcc42289b6bfe02ed020dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:50:29.332922 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.334703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.336290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1358/2747] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:50:29 2022] DELETE /identity/v3/projects/6842f55d3dcc42289b6bfe02ed020dc4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:50:29.340648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:29.342052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:29.355841 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493817d6-dba2-4084-b069-aef726b94e82 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:29.368348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:50:29.418524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:29.419161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:29.419440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:29.419709 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] PATH_INFO: `/v3/users/532f2f52948a4074abff9ea07cf09f47` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:29.420898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=532f2f52948a4074abff9ea07cf09f47)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:29.424557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:29.593996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532f2f52948a4074abff9ea07cf09f47'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.594610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532f2f52948a4074abff9ea07cf09f47'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.601737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532f2f52948a4074abff9ea07cf09f47'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.686473 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532f2f52948a4074abff9ea07cf09f47'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.687094 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532f2f52948a4074abff9ea07cf09f47'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:29.706323 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1390/2748] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:50:29 2022] DELETE /identity/v3/users/532f2f52948a4074abff9ea07cf09f47 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:50:29.973981 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:50:29.975631 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:50:29.990580 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdcf4c-3381-464a-bc42-a91b2c291e6f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:50:30.049397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:50:30.050076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:50:30.050304 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:50:30.050589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] PATH_INFO: `/v3/projects/acc55cc2d1e34e70aacabed458a3b107` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:50:30.051835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc55cc2d1e34e70aacabed458a3b107)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:50:30.053660 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:50:30.197178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc55cc2d1e34e70aacabed458a3b107', 'request_id': 'req-776d098d-bb6f-44f2-9894-230d0a038228'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:30.203906 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc55cc2d1e34e70aacabed458a3b107', 'request_id': 'req-776d098d-bb6f-44f2-9894-230d0a038228'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:30.210446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc55cc2d1e34e70aacabed458a3b107', 'request_id': 'req-776d098d-bb6f-44f2-9894-230d0a038228'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:30.298601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] The token cache is being invalidate because project acc55cc2d1e34e70aacabed458a3b107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:50:30.298894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:30.299554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:50:30.301091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1359/2749] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:50:29 2022] DELETE /identity/v3/projects/acc55cc2d1e34e70aacabed458a3b107 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:50:39.702593 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:50:40.295348 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=68,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:16.260872 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:16.262290 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:16.321764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a76711-ad60-4b8c-9875-989836d5cdaa admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:16.370543 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:16.371145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:16.371428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:16.371702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:16.373493 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:16.437625 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:16.439138 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:16.449068 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1391/2750] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:51:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:51:26.444806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=68 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:33.110891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:33.113098 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:33.170319 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d098d-bb6f-44f2-9894-230d0a038228 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:33.174002 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:33.175239 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:33.175526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:33.175526 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:33.177594 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:33.240706 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:33.242144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:33.251630 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1360/2751] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:51:36.989056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:36.990672 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:37.004708 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4f283-31c0-42e8-bdde-e6dd73652dec service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:37.009284 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:37.009915 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:37.010202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:37.010545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] PATH_INFO: `/v3/users/33835c5512df4c80a9b7cb9712a5904f` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:37.011610 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33835c5512df4c80a9b7cb9712a5904f)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:37.013559 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:37.256496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33835c5512df4c80a9b7cb9712a5904f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.257903 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33835c5512df4c80a9b7cb9712a5904f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.264477 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33835c5512df4c80a9b7cb9712a5904f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.325979 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33835c5512df4c80a9b7cb9712a5904f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.326702 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-821946cf-434d-47a7-90d2-a8a1a806f2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33835c5512df4c80a9b7cb9712a5904f'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.348621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1392/2752] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:51:36 2022] DELETE /identity/v3/users/33835c5512df4c80a9b7cb9712a5904f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:37.573677 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:37.586554 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3aa09-c8d6-4238-83b8-24e5649489de service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:37.650151 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:37.650789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:37.651067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:37.651341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] PATH_INFO: `/v3/projects/ca298d9092ad4a9489f1624d62b2ac1d` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:37.652533 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca298d9092ad4a9489f1624d62b2ac1d)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:37.654493 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:37.789109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca298d9092ad4a9489f1624d62b2ac1d', 'request_id': 'req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.791069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:37.794908 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca298d9092ad4a9489f1624d62b2ac1d', 'request_id': 'req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.801234 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca298d9092ad4a9489f1624d62b2ac1d', 'request_id': 'req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.872948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] The token cache is being invalidate because project ca298d9092ad4a9489f1624d62b2ac1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:51:37.873177 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.873827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ba35c65a-75e3-4a9e-8b1a-d892833bd9d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:37.875424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1361/2753] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:51:37 2022] DELETE /identity/v3/projects/ca298d9092ad4a9489f1624d62b2ac1d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:37.880442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb667e1-a25c-45a0-927e-e643580b3193 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:37.883006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb667e1-a25c-45a0-927e-e643580b3193 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:37.883006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb667e1-a25c-45a0-927e-e643580b3193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:37.901754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9cb667e1-a25c-45a0-927e-e643580b3193 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:37.902985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9cb667e1-a25c-45a0-927e-e643580b3193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:37.976763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1393/2754] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:51:37.985686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb8d1-8a97-4314-b2b6-3bb82fa9cbda None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:37.985974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb8d1-8a97-4314-b2b6-3bb82fa9cbda None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:37.991547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb8d1-8a97-4314-b2b6-3bb82fa9cbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.007113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-3c6cb8d1-8a97-4314-b2b6-3bb82fa9cbda None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:38.008409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3c6cb8d1-8a97-4314-b2b6-3bb82fa9cbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:38.016434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1362/2755] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:51:38.021486 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb667e1-a25c-45a0-927e-e643580b3193 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:38.031261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb667e1-a25c-45a0-927e-e643580b3193 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:38.035886 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:38.037382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.037382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.037382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.037732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:51:38.038032 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:38.039624 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:38.051285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1394/2756] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:51:38.055620 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cb8d1-8a97-4314-b2b6-3bb82fa9cbda None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:38.065454 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cb8d1-8a97-4314-b2b6-3bb82fa9cbda None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:38.070612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b57aab-1f0a-4433-b347-63364f988d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:38.071209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b57aab-1f0a-4433-b347-63364f988d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.071492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b57aab-1f0a-4433-b347-63364f988d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.071766 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b57aab-1f0a-4433-b347-63364f988d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.072434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b57aab-1f0a-4433-b347-63364f988d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:38.074333 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b57aab-1f0a-4433-b347-63364f988d1f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:38.191149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1363/2757] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:51:38 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:51:38.195562 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:38.207263 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9292eb8-53c4-4cdc-bc2e-55ee0c538a0f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:38.262511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98c0e0-ea0e-4aeb-9e58-da99f9f2cc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:38.263103 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98c0e0-ea0e-4aeb-9e58-da99f9f2cc78 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.263381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98c0e0-ea0e-4aeb-9e58-da99f9f2cc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.263652 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98c0e0-ea0e-4aeb-9e58-da99f9f2cc78 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.264305 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98c0e0-ea0e-4aeb-9e58-da99f9f2cc78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:38.266203 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98c0e0-ea0e-4aeb-9e58-da99f9f2cc78 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:38.274657 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:38.366489 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1395/2758] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:51:38 2022] POST /identity/v3/users => generated 282 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:51:38.370673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b57aab-1f0a-4433-b347-63364f988d1f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:38.382089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b57aab-1f0a-4433-b347-63364f988d1f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:38.386909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8193-c6a2-42e1-9f83-64553314e3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:38.387518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8193-c6a2-42e1-9f83-64553314e3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.387796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8193-c6a2-42e1-9f83-64553314e3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.388066 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8193-c6a2-42e1-9f83-64553314e3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.388747 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d8193-c6a2-42e1-9f83-64553314e3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:38.390343 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d8193-c6a2-42e1-9f83-64553314e3ab admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:38.401686 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1364/2759] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:51:38.405838 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98c0e0-ea0e-4aeb-9e58-da99f9f2cc78 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:38.416654 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98c0e0-ea0e-4aeb-9e58-da99f9f2cc78 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:38.421324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e33883-d29e-4617-aa44-32a2a1bd1bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:38.421920 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e33883-d29e-4617-aa44-32a2a1bd1bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.422201 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e33883-d29e-4617-aa44-32a2a1bd1bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.422474 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e33883-d29e-4617-aa44-32a2a1bd1bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.423124 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e33883-d29e-4617-aa44-32a2a1bd1bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:38.425338 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e33883-d29e-4617-aa44-32a2a1bd1bc1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:38.437396 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1396/2760] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:51:38.442329 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8193-c6a2-42e1-9f83-64553314e3ab admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:38.452563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8193-c6a2-42e1-9f83-64553314e3ab admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:38.457206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036efb89-f7a5-43f8-b5a1-1c8f8cd88fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:38.457850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036efb89-f7a5-43f8-b5a1-1c8f8cd88fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.458131 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036efb89-f7a5-43f8-b5a1-1c8f8cd88fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.458463 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036efb89-f7a5-43f8-b5a1-1c8f8cd88fa8 admin admin] PATH_INFO: `/v3/projects/7d3d039263c54c06950b3c85c4a78bcf/users/f48521a70a32421abb83e350ae771fc3/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.473409 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036efb89-f7a5-43f8-b5a1-1c8f8cd88fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d3d039263c54c06950b3c85c4a78bcf, user_id=f48521a70a32421abb83e350ae771fc3, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:38.476045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036efb89-f7a5-43f8-b5a1-1c8f8cd88fa8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:38.541761 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1365/2761] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:51:38 2022] PUT /identity/v3/projects/7d3d039263c54c06950b3c85c4a78bcf/users/f48521a70a32421abb83e350ae771fc3/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:38.553297 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436d3f1-a571-4826-8f3b-8afd83b2b815 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.553588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436d3f1-a571-4826-8f3b-8afd83b2b815 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.553876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436d3f1-a571-4826-8f3b-8afd83b2b815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.583440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9436d3f1-a571-4826-8f3b-8afd83b2b815 None None] MFA Rules not processed for user `f48521a70a32421abb83e350ae771fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:38.584696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9436d3f1-a571-4826-8f3b-8afd83b2b815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:38.660206 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1397/2762] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:51:38.664718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036efb89-f7a5-43f8-b5a1-1c8f8cd88fa8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:38.727079 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036efb89-f7a5-43f8-b5a1-1c8f8cd88fa8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:38.731733 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01300db-8879-46ef-8f2a-4b8b23bbaae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:38.732366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01300db-8879-46ef-8f2a-4b8b23bbaae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.732654 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01300db-8879-46ef-8f2a-4b8b23bbaae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.732931 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01300db-8879-46ef-8f2a-4b8b23bbaae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.733570 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01300db-8879-46ef-8f2a-4b8b23bbaae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:38.735483 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01300db-8879-46ef-8f2a-4b8b23bbaae9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:38.852844 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1366/2763] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:51:38 2022] POST /identity/v3/projects => generated 363 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:51:38.858558 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436d3f1-a571-4826-8f3b-8afd83b2b815 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:38.870749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436d3f1-a571-4826-8f3b-8afd83b2b815 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:38.937747 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2696f3-1d66-495e-b77b-eb230349f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:38.938362 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2696f3-1d66-495e-b77b-eb230349f94e admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:38.938643 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2696f3-1d66-495e-b77b-eb230349f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:38.938917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2696f3-1d66-495e-b77b-eb230349f94e admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:38.939560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2696f3-1d66-495e-b77b-eb230349f94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:38.942109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2696f3-1d66-495e-b77b-eb230349f94e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:39.035680 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1398/2764] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:51:38 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:51:39.039788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01300db-8879-46ef-8f2a-4b8b23bbaae9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:39.051714 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01300db-8879-46ef-8f2a-4b8b23bbaae9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:39.056939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ec688-8972-48fc-ae72-4e431bc275f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:39.057577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ec688-8972-48fc-ae72-4e431bc275f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:39.057863 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ec688-8972-48fc-ae72-4e431bc275f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:39.058140 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ec688-8972-48fc-ae72-4e431bc275f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:39.058822 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ec688-8972-48fc-ae72-4e431bc275f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:39.060490 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ec688-8972-48fc-ae72-4e431bc275f2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:39.071752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1367/2765] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:51:39.076123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2696f3-1d66-495e-b77b-eb230349f94e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:39.087221 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2696f3-1d66-495e-b77b-eb230349f94e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:39.091905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99112748-2fad-4982-bc43-b64c3e8e524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:39.092584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99112748-2fad-4982-bc43-b64c3e8e524d admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:39.092868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99112748-2fad-4982-bc43-b64c3e8e524d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:39.093142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99112748-2fad-4982-bc43-b64c3e8e524d admin admin] PATH_INFO: `/v3/projects/07892da9ff15493db1707b187aeebd68/users/1d3e4a7dc12c4b2aa6aa07b5eaa2f61a/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:39.108703 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99112748-2fad-4982-bc43-b64c3e8e524d admin admin] RBAC: Authorizing `identity:create_grant(project_id=07892da9ff15493db1707b187aeebd68, user_id=1d3e4a7dc12c4b2aa6aa07b5eaa2f61a, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:39.111401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99112748-2fad-4982-bc43-b64c3e8e524d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:39.188818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1399/2766] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:51:39 2022] PUT /identity/v3/projects/07892da9ff15493db1707b187aeebd68/users/1d3e4a7dc12c4b2aa6aa07b5eaa2f61a/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:39.192941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ec688-8972-48fc-ae72-4e431bc275f2 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:39.203202 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ec688-8972-48fc-ae72-4e431bc275f2 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:39.244222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:39.261051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b41482-64bf-462b-925c-6db61e2f39af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:39.261703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b41482-64bf-462b-925c-6db61e2f39af admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:39.261994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b41482-64bf-462b-925c-6db61e2f39af admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:39.262275 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b41482-64bf-462b-925c-6db61e2f39af admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:39.262942 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b41482-64bf-462b-925c-6db61e2f39af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:39.264561 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b41482-64bf-462b-925c-6db61e2f39af admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:39.276256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1368/2767] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:51:39.279812 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99112748-2fad-4982-bc43-b64c3e8e524d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:39.291399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99112748-2fad-4982-bc43-b64c3e8e524d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:39.295994 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:39.296711 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:39.296996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:39.297273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] PATH_INFO: `/v3/projects/07892da9ff15493db1707b187aeebd68/users/1d3e4a7dc12c4b2aa6aa07b5eaa2f61a/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:39.299211 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=07892da9ff15493db1707b187aeebd68, user_id=1d3e4a7dc12c4b2aa6aa07b5eaa2f61a, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:39.302410 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:39.313219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1400/2768] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:51:39 2022] PUT /identity/v3/projects/07892da9ff15493db1707b187aeebd68/users/1d3e4a7dc12c4b2aa6aa07b5eaa2f61a/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:39.324027 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73b615-805c-463b-8ae6-04de68814c7e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:39.324354 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73b615-805c-463b-8ae6-04de68814c7e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:39.324639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73b615-805c-463b-8ae6-04de68814c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:39.354530 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-4b73b615-805c-463b-8ae6-04de68814c7e None None] MFA Rules not processed for user `1d3e4a7dc12c4b2aa6aa07b5eaa2f61a`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:39.355755 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4b73b615-805c-463b-8ae6-04de68814c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:39.426163 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1369/2769] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:51:39.437507 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:39.439193 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:39.463798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:39.499966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edff73-1469-470e-b218-42a087292a2a admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:39.504261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b7c14-40ba-49cb-b14c-7494762594b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:39.504913 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b7c14-40ba-49cb-b14c-7494762594b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:39.508091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b7c14-40ba-49cb-b14c-7494762594b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:39.508091 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b7c14-40ba-49cb-b14c-7494762594b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:39.514805 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b7c14-40ba-49cb-b14c-7494762594b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:39.516246 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b7c14-40ba-49cb-b14c-7494762594b7 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:39.526444 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1401/2770] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:51:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:51:39.707433 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73b615-805c-463b-8ae6-04de68814c7e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:39.717504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73b615-805c-463b-8ae6-04de68814c7e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:39.721146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea613f-15f4-4cbe-af5d-e35a55b3328a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:39.721816 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea613f-15f4-4cbe-af5d-e35a55b3328a service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:39.722092 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea613f-15f4-4cbe-af5d-e35a55b3328a service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:39.722364 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea613f-15f4-4cbe-af5d-e35a55b3328a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:39.778839 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea613f-15f4-4cbe-af5d-e35a55b3328a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:39.780306 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea613f-15f4-4cbe-af5d-e35a55b3328a service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:39.789350 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1370/2771] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:51:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:51:43.216428 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b7c14-40ba-49cb-b14c-7494762594b7 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:43.281183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b7c14-40ba-49cb-b14c-7494762594b7 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:43.286460 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:43.287082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:43.287359 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:43.287630 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] PATH_INFO: `/v3/users/f48521a70a32421abb83e350ae771fc3` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:43.288765 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48521a70a32421abb83e350ae771fc3)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:43.290834 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:43.472259 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48521a70a32421abb83e350ae771fc3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:43.472883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48521a70a32421abb83e350ae771fc3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:43.480370 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48521a70a32421abb83e350ae771fc3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:43.541991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48521a70a32421abb83e350ae771fc3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:43.542613 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48521a70a32421abb83e350ae771fc3'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:43.561155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1402/2772] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:51:43 2022] DELETE /identity/v3/users/f48521a70a32421abb83e350ae771fc3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:43.565872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89eaea5-18fe-4e5b-9c85-37b8db4dceec None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:43.566385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89eaea5-18fe-4e5b-9c85-37b8db4dceec None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:43.566518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89eaea5-18fe-4e5b-9c85-37b8db4dceec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:43.587606 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-e89eaea5-18fe-4e5b-9c85-37b8db4dceec None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:43.588866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e89eaea5-18fe-4e5b-9c85-37b8db4dceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:43.653397 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1371/2773] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:51:43.663988 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:43.665433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:43.712626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:43.725119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136a47f-d1a2-497a-aff9-fd96025e98c5 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:43.728791 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e9980-84bf-487f-aca5-3306ca58849a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:43.729394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e9980-84bf-487f-aca5-3306ca58849a service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:43.729753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e9980-84bf-487f-aca5-3306ca58849a service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:43.731854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e9980-84bf-487f-aca5-3306ca58849a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:43.739638 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e9980-84bf-487f-aca5-3306ca58849a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:43.741119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e9980-84bf-487f-aca5-3306ca58849a service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:43.751324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1403/2774] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:51:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:51:44.177887 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89eaea5-18fe-4e5b-9c85-37b8db4dceec None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:44.179410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-e89eaea5-18fe-4e5b-9c85-37b8db4dceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:44.186505 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:44.192540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89eaea5-18fe-4e5b-9c85-37b8db4dceec None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:44.197610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:44.198252 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:44.198556 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:44.198830 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] PATH_INFO: `/v3/projects/7d3d039263c54c06950b3c85c4a78bcf` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:44.199976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d3d039263c54c06950b3c85c4a78bcf)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:44.201929 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:44.354395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d3d039263c54c06950b3c85c4a78bcf', 'request_id': 'req-b6fd5fce-649d-4dd9-b863-8268d6f9a999'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.360233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d3d039263c54c06950b3c85c4a78bcf', 'request_id': 'req-b6fd5fce-649d-4dd9-b863-8268d6f9a999'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.367296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d3d039263c54c06950b3c85c4a78bcf', 'request_id': 'req-b6fd5fce-649d-4dd9-b863-8268d6f9a999'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.446139 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] The token cache is being invalidate because project 7d3d039263c54c06950b3c85c4a78bcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:51:44.446426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.447050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.448579 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1372/2775] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:51:44 2022] DELETE /identity/v3/projects/7d3d039263c54c06950b3c85c4a78bcf => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:44.452912 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e9980-84bf-487f-aca5-3306ca58849a service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:44.462803 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e9980-84bf-487f-aca5-3306ca58849a service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:44.521895 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:44.522518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:44.522793 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:44.523065 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] PATH_INFO: `/v3/users/1d3e4a7dc12c4b2aa6aa07b5eaa2f61a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:44.524198 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d3e4a7dc12c4b2aa6aa07b5eaa2f61a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:44.526130 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:44.751566 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d3e4a7dc12c4b2aa6aa07b5eaa2f61a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.752296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d3e4a7dc12c4b2aa6aa07b5eaa2f61a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.759458 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d3e4a7dc12c4b2aa6aa07b5eaa2f61a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.816629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d3e4a7dc12c4b2aa6aa07b5eaa2f61a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.817261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c1996e4e-498a-4c48-afa5-7848d76bba63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d3e4a7dc12c4b2aa6aa07b5eaa2f61a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:44.836447 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1404/2776] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:51:44 2022] DELETE /identity/v3/users/1d3e4a7dc12c4b2aa6aa07b5eaa2f61a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:45.241749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:45.243236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:45.258708 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd5fce-649d-4dd9-b863-8268d6f9a999 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:45.317518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:45.318148 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:45.318742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:45.319017 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] PATH_INFO: `/v3/projects/07892da9ff15493db1707b187aeebd68` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:45.320205 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=07892da9ff15493db1707b187aeebd68)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:45.322110 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:45.425011 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07892da9ff15493db1707b187aeebd68', 'request_id': 'req-3916fa10-1692-42db-82e1-8cf9bddaa9ce'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:45.440322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07892da9ff15493db1707b187aeebd68', 'request_id': 'req-3916fa10-1692-42db-82e1-8cf9bddaa9ce'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:45.440322 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07892da9ff15493db1707b187aeebd68', 'request_id': 'req-3916fa10-1692-42db-82e1-8cf9bddaa9ce'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:45.536589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] The token cache is being invalidate because project 07892da9ff15493db1707b187aeebd68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:51:45.536883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:45.537510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-3916fa10-1692-42db-82e1-8cf9bddaa9ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:51:45.539065 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1373/2777] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:51:45 2022] DELETE /identity/v3/projects/07892da9ff15493db1707b187aeebd68 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:51:45.543974 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c88ff-b8bc-448f-bd79-7204953d628f None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:45.544484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c88ff-b8bc-448f-bd79-7204953d628f None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:45.544584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c88ff-b8bc-448f-bd79-7204953d628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:45.564572 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-9f3c88ff-b8bc-448f-bd79-7204953d628f None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:45.566342 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9f3c88ff-b8bc-448f-bd79-7204953d628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:45.629890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1405/2778] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:51:45.637898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d331b-1856-45e6-bf04-8e2c704a5fe3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:45.638178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d331b-1856-45e6-bf04-8e2c704a5fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:45.638461 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d331b-1856-45e6-bf04-8e2c704a5fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:45.659407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-b65d331b-1856-45e6-bf04-8e2c704a5fe3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:45.660679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-b65d331b-1856-45e6-bf04-8e2c704a5fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:45.668866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1374/2779] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:51:45.673261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c88ff-b8bc-448f-bd79-7204953d628f None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:45.679308 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:45.683026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c88ff-b8bc-448f-bd79-7204953d628f None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:45.687644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:45.688294 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:45.688580 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:45.694052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:45.694052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:51:45.694052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:45.694052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:45.703847 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1406/2780] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:51:45.709107 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d331b-1856-45e6-bf04-8e2c704a5fe3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:45.718501 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d331b-1856-45e6-bf04-8e2c704a5fe3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:45.723359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e520bd6-1e91-4321-b207-a98ccefaad72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:45.723952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e520bd6-1e91-4321-b207-a98ccefaad72 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:45.724268 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e520bd6-1e91-4321-b207-a98ccefaad72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:45.724548 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e520bd6-1e91-4321-b207-a98ccefaad72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:45.726127 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e520bd6-1e91-4321-b207-a98ccefaad72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:45.727051 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e520bd6-1e91-4321-b207-a98ccefaad72 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:45.732438 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:45.808718 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1375/2781] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:51:45 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:51:45.812640 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:45.823950 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cd976-7eff-433a-8ec2-00c1f548771e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:45.881258 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21de40e-3165-4000-9ccd-80f73a7733eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:45.881860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21de40e-3165-4000-9ccd-80f73a7733eb admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:45.882142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21de40e-3165-4000-9ccd-80f73a7733eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:45.882417 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21de40e-3165-4000-9ccd-80f73a7733eb admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:45.883044 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21de40e-3165-4000-9ccd-80f73a7733eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:45.885026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21de40e-3165-4000-9ccd-80f73a7733eb admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:45.970982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1407/2782] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:51:45 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:51:45.975082 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e520bd6-1e91-4321-b207-a98ccefaad72 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:45.985496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e520bd6-1e91-4321-b207-a98ccefaad72 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:45.990936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b20480-585b-474a-884c-a4bf0273a66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:45.991527 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20480-585b-474a-884c-a4bf0273a66f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:45.991810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20480-585b-474a-884c-a4bf0273a66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:45.992085 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20480-585b-474a-884c-a4bf0273a66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:45.992810 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b20480-585b-474a-884c-a4bf0273a66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:45.994404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b20480-585b-474a-884c-a4bf0273a66f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.005212 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1376/2783] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:51:46.009162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21de40e-3165-4000-9ccd-80f73a7733eb admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.019249 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21de40e-3165-4000-9ccd-80f73a7733eb admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:46.023876 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32f80d-cf0b-44d6-b493-57e94b51c18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:46.024502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32f80d-cf0b-44d6-b493-57e94b51c18a admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.024789 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32f80d-cf0b-44d6-b493-57e94b51c18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.025061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32f80d-cf0b-44d6-b493-57e94b51c18a admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.025707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32f80d-cf0b-44d6-b493-57e94b51c18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:46.027341 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32f80d-cf0b-44d6-b493-57e94b51c18a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.038108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1408/2784] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:51:46.042836 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b20480-585b-474a-884c-a4bf0273a66f admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.053695 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b20480-585b-474a-884c-a4bf0273a66f admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:46.058405 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20ce5f-1938-4df4-842c-02dc33960cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:46.059109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ce5f-1938-4df4-842c-02dc33960cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.059341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ce5f-1938-4df4-842c-02dc33960cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.059615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20ce5f-1938-4df4-842c-02dc33960cbb admin admin] PATH_INFO: `/v3/projects/0cab5a58ed5240c6877afd1cdfe2c702/users/f4adc39d93b14d8cb89c1af5f24d0d08/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.075064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20ce5f-1938-4df4-842c-02dc33960cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cab5a58ed5240c6877afd1cdfe2c702, user_id=f4adc39d93b14d8cb89c1af5f24d0d08, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:46.077751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20ce5f-1938-4df4-842c-02dc33960cbb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.154568 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1377/2785] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:51:46 2022] PUT /identity/v3/projects/0cab5a58ed5240c6877afd1cdfe2c702/users/f4adc39d93b14d8cb89c1af5f24d0d08/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:46.166064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597088e-9696-4621-9138-8e4f2e5fd82a None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.166385 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597088e-9696-4621-9138-8e4f2e5fd82a None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.166639 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597088e-9696-4621-9138-8e4f2e5fd82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.196173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5597088e-9696-4621-9138-8e4f2e5fd82a None None] MFA Rules not processed for user `f4adc39d93b14d8cb89c1af5f24d0d08`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:46.197462 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5597088e-9696-4621-9138-8e4f2e5fd82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:46.261529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1409/2786] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:51:46.266036 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20ce5f-1938-4df4-842c-02dc33960cbb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.329359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20ce5f-1938-4df4-842c-02dc33960cbb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:46.334465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8aa018-a37e-4c38-9042-31114b8c95a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:46.335063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aa018-a37e-4c38-9042-31114b8c95a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.335345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aa018-a37e-4c38-9042-31114b8c95a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.335619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8aa018-a37e-4c38-9042-31114b8c95a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.336277 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8aa018-a37e-4c38-9042-31114b8c95a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:46.338170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8aa018-a37e-4c38-9042-31114b8c95a1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.418481 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1378/2787] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:51:46 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:51:46.422754 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597088e-9696-4621-9138-8e4f2e5fd82a None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.434917 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597088e-9696-4621-9138-8e4f2e5fd82a None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:46.492696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4017eb-5502-45d4-b996-50b86b23984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:46.493304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4017eb-5502-45d4-b996-50b86b23984b admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.493595 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4017eb-5502-45d4-b996-50b86b23984b admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.493910 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4017eb-5502-45d4-b996-50b86b23984b admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.494529 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4017eb-5502-45d4-b996-50b86b23984b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:46.496471 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4017eb-5502-45d4-b996-50b86b23984b admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.504546 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:46.577126 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1410/2788] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:51:46 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:51:46.581878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8aa018-a37e-4c38-9042-31114b8c95a1 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.592221 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8aa018-a37e-4c38-9042-31114b8c95a1 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:46.598112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace57d7-46ff-4ad1-92f7-a71a4a168f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:46.598112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace57d7-46ff-4ad1-92f7-a71a4a168f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.598351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace57d7-46ff-4ad1-92f7-a71a4a168f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.598598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace57d7-46ff-4ad1-92f7-a71a4a168f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.599235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace57d7-46ff-4ad1-92f7-a71a4a168f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:46.600865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace57d7-46ff-4ad1-92f7-a71a4a168f22 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.612752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1379/2789] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:51:46.616985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4017eb-5502-45d4-b996-50b86b23984b admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.627254 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4017eb-5502-45d4-b996-50b86b23984b admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:46.632011 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f891ed-0dcf-4c17-b5a4-e755113a2a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:46.632753 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f891ed-0dcf-4c17-b5a4-e755113a2a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.633038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f891ed-0dcf-4c17-b5a4-e755113a2a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.633314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f891ed-0dcf-4c17-b5a4-e755113a2a51 admin admin] PATH_INFO: `/v3/projects/060e57a4eaec4c10a48d2863227a4564/users/8a65b1be4f7b404988e2bb67f421d31d/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.649160 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f891ed-0dcf-4c17-b5a4-e755113a2a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060e57a4eaec4c10a48d2863227a4564, user_id=8a65b1be4f7b404988e2bb67f421d31d, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:46.651870 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f891ed-0dcf-4c17-b5a4-e755113a2a51 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.714645 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1411/2790] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:51:46 2022] PUT /identity/v3/projects/060e57a4eaec4c10a48d2863227a4564/users/8a65b1be4f7b404988e2bb67f421d31d/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:46.718612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace57d7-46ff-4ad1-92f7-a71a4a168f22 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.729641 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace57d7-46ff-4ad1-92f7-a71a4a168f22 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:46.787291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9194c0-35a6-4420-a5ab-6eafac2a3aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:46.787920 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9194c0-35a6-4420-a5ab-6eafac2a3aee admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.788230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9194c0-35a6-4420-a5ab-6eafac2a3aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.788514 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9194c0-35a6-4420-a5ab-6eafac2a3aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.789156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9194c0-35a6-4420-a5ab-6eafac2a3aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:46.790753 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9194c0-35a6-4420-a5ab-6eafac2a3aee admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.800717 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1380/2791] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:51:46.805056 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f891ed-0dcf-4c17-b5a4-e755113a2a51 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.815769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f891ed-0dcf-4c17-b5a4-e755113a2a51 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:46.820758 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:46.821404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.822066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.822345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] PATH_INFO: `/v3/projects/060e57a4eaec4c10a48d2863227a4564/users/8a65b1be4f7b404988e2bb67f421d31d/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.824361 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060e57a4eaec4c10a48d2863227a4564, user_id=8a65b1be4f7b404988e2bb67f421d31d, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:46.827039 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:46.838500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1412/2792] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:51:46 2022] PUT /identity/v3/projects/060e57a4eaec4c10a48d2863227a4564/users/8a65b1be4f7b404988e2bb67f421d31d/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:51:46.850575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e8a3-4981-4b5c-b923-dbe30fc7c215 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:46.850867 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e8a3-4981-4b5c-b923-dbe30fc7c215 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:46.851146 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e8a3-4981-4b5c-b923-dbe30fc7c215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:46.878451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-ed21e8a3-4981-4b5c-b923-dbe30fc7c215 None None] MFA Rules not processed for user `8a65b1be4f7b404988e2bb67f421d31d`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:51:46.879681 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ed21e8a3-4981-4b5c-b923-dbe30fc7c215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:46.888912 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:51:46.949759 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1381/2793] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:51:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:51:46.962394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:46.964229 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:47.023552 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2b382-adea-4a19-ab2d-611bb98b4398 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:47.027395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676da18-db23-4bec-b0a0-968d48f14b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:47.028008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676da18-db23-4bec-b0a0-968d48f14b37 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:47.028328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676da18-db23-4bec-b0a0-968d48f14b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:47.028605 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676da18-db23-4bec-b0a0-968d48f14b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:47.087107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676da18-db23-4bec-b0a0-968d48f14b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:47.088644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676da18-db23-4bec-b0a0-968d48f14b37 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:47.098142 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1413/2794] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:51:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:51:53.326109 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21e8a3-4981-4b5c-b923-dbe30fc7c215 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:51:53.327639 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-ed21e8a3-4981-4b5c-b923-dbe30fc7c215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:51:53.341181 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21e8a3-4981-4b5c-b923-dbe30fc7c215 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:51:53.349054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef383e3-464b-4f33-b2e1-1ed88c923e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:51:53.349671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef383e3-464b-4f33-b2e1-1ed88c923e7e service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:51:53.349955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef383e3-464b-4f33-b2e1-1ed88c923e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:51:53.350230 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef383e3-464b-4f33-b2e1-1ed88c923e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:51:53.362869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef383e3-464b-4f33-b2e1-1ed88c923e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:51:53.364379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef383e3-464b-4f33-b2e1-1ed88c923e7e service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:51:53.374432 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1382/2795] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:51:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:51:57.093446 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:INSERT=5,keystone:SELECT=65 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:03.370037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=30 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:20.358712 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676da18-db23-4bec-b0a0-968d48f14b37 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:20.421037 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676da18-db23-4bec-b0a0-968d48f14b37 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:20.425527 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:20.426162 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:20.426465 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:20.426728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] PATH_INFO: `/v3/users/f4adc39d93b14d8cb89c1af5f24d0d08` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:20.428053 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4adc39d93b14d8cb89c1af5f24d0d08)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:20.430095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:20.660173 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4adc39d93b14d8cb89c1af5f24d0d08'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:20.661425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4adc39d93b14d8cb89c1af5f24d0d08'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:20.668197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4adc39d93b14d8cb89c1af5f24d0d08'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:20.753713 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4adc39d93b14d8cb89c1af5f24d0d08'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:20.754344 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4adc39d93b14d8cb89c1af5f24d0d08'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:20.778445 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1414/2796] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:52:20 2022] DELETE /identity/v3/users/f4adc39d93b14d8cb89c1af5f24d0d08 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:20.783037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544ce1a-267e-4676-b780-94938390e1d3 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:20.783515 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544ce1a-267e-4676-b780-94938390e1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:20.783628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544ce1a-267e-4676-b780-94938390e1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:20.802598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d544ce1a-267e-4676-b780-94938390e1d3 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:20.803807 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d544ce1a-267e-4676-b780-94938390e1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:20.864169 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1383/2797] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:20.874155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:20.875569 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:20.937185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970f9af-cf1f-4c53-9437-f4a964bacf5f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:20.940905 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3707ec-3fcc-4120-80f1-4e070dddab85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:20.941512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3707ec-3fcc-4120-80f1-4e070dddab85 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:20.941798 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3707ec-3fcc-4120-80f1-4e070dddab85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:20.942077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3707ec-3fcc-4120-80f1-4e070dddab85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:20.951490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3707ec-3fcc-4120-80f1-4e070dddab85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:20.952957 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3707ec-3fcc-4120-80f1-4e070dddab85 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:20.963382 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1415/2798] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:52:21.459054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544ce1a-267e-4676-b780-94938390e1d3 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:21.461200 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d544ce1a-267e-4676-b780-94938390e1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:21.473602 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544ce1a-267e-4676-b780-94938390e1d3 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:21.479124 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:21.479765 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:21.480643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:21.480643 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] PATH_INFO: `/v3/projects/0cab5a58ed5240c6877afd1cdfe2c702` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:21.481589 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cab5a58ed5240c6877afd1cdfe2c702)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:21.483519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:21.605777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cab5a58ed5240c6877afd1cdfe2c702', 'request_id': 'req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:21.611367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cab5a58ed5240c6877afd1cdfe2c702', 'request_id': 'req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:21.617960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cab5a58ed5240c6877afd1cdfe2c702', 'request_id': 'req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:21.667049 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] The token cache is being invalidate because project 0cab5a58ed5240c6877afd1cdfe2c702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:52:21.667366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:21.667995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:21.669541 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1384/2799] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:52:21 2022] DELETE /identity/v3/projects/0cab5a58ed5240c6877afd1cdfe2c702 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:21.673647 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3707ec-3fcc-4120-80f1-4e070dddab85 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:21.684089 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3707ec-3fcc-4120-80f1-4e070dddab85 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:21.703021 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:21.741435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:21.741975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:21.742264 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:21.742542 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] PATH_INFO: `/v3/users/8a65b1be4f7b404988e2bb67f421d31d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:21.743686 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a65b1be4f7b404988e2bb67f421d31d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:21.745646 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:21.944792 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a65b1be4f7b404988e2bb67f421d31d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:21.945459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a65b1be4f7b404988e2bb67f421d31d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:21.952879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a65b1be4f7b404988e2bb67f421d31d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:22.073557 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a65b1be4f7b404988e2bb67f421d31d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:22.074188 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-c535bb65-c221-4fde-8753-8c6f77b8c33f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a65b1be4f7b404988e2bb67f421d31d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:22.092833 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1416/2800] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:52:21 2022] DELETE /identity/v3/users/8a65b1be4f7b404988e2bb67f421d31d => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:22.487410 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:22.489470 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:22.508106 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb44210-ba6c-4cda-979d-74fb8e2b28ec admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:22.585948 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:22.586604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:22.586891 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:22.587199 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] PATH_INFO: `/v3/projects/060e57a4eaec4c10a48d2863227a4564` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:22.588466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060e57a4eaec4c10a48d2863227a4564)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:22.590411 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:22.675496 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:22.694250 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060e57a4eaec4c10a48d2863227a4564', 'request_id': 'req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:22.699737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060e57a4eaec4c10a48d2863227a4564', 'request_id': 'req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:22.705921 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060e57a4eaec4c10a48d2863227a4564', 'request_id': 'req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:22.816063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] The token cache is being invalidate because project 060e57a4eaec4c10a48d2863227a4564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81685) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:52:22.816404 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:22.817048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-74821b49-9a54-4ee9-99d6-bb452ab8d5b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:22.819751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1385/2801] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:52:22 2022] DELETE /identity/v3/projects/060e57a4eaec4c10a48d2863227a4564 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:22.824642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2af54-4506-4320-b98f-6b3b1d296ba4 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:22.825177 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2af54-4506-4320-b98f-6b3b1d296ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:22.825298 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2af54-4506-4320-b98f-6b3b1d296ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:22.845506 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-b6d2af54-4506-4320-b98f-6b3b1d296ba4 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:22.846769 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-b6d2af54-4506-4320-b98f-6b3b1d296ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:22.919707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1417/2802] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:22.928983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c706a-a16c-4974-8fe0-d741cf7bdc8a None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:22.929206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c706a-a16c-4974-8fe0-d741cf7bdc8a None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:22.929484 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c706a-a16c-4974-8fe0-d741cf7bdc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:22.949539 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-847c706a-a16c-4974-8fe0-d741cf7bdc8a None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:22.950751 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-847c706a-a16c-4974-8fe0-d741cf7bdc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:22.959498 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1386/2803] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:22.963705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2af54-4506-4320-b98f-6b3b1d296ba4 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:22.973571 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2af54-4506-4320-b98f-6b3b1d296ba4 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:22.979095 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:22.979692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:22.979981 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:22.980315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:22.980968 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81686) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:52:22.981274 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:22.982880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:22.995134 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1418/2804] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:52:22.999377 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c706a-a16c-4974-8fe0-d741cf7bdc8a None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:23.008966 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c706a-a16c-4974-8fe0-d741cf7bdc8a None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:23.013805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0d52e-c5f3-41a6-9658-422812667a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:23.014401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0d52e-c5f3-41a6-9658-422812667a57 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:23.014685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0d52e-c5f3-41a6-9658-422812667a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:23.014961 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0d52e-c5f3-41a6-9658-422812667a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:23.015621 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0d52e-c5f3-41a6-9658-422812667a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:23.017558 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0d52e-c5f3-41a6-9658-422812667a57 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:23.133080 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1387/2805] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:52:22 2022] POST /identity/v3/projects => generated 401 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:23.137459 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:23.148498 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ad608-96e3-4379-bc49-906e85fe06a9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:23.175155 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:23.207573 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2e432-d547-4bb0-9980-e9be79e1b1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:23.208107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e432-d547-4bb0-9980-e9be79e1b1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:23.208937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e432-d547-4bb0-9980-e9be79e1b1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:23.208937 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2e432-d547-4bb0-9980-e9be79e1b1a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:23.209484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2e432-d547-4bb0-9980-e9be79e1b1a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:23.211432 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c2e432-d547-4bb0-9980-e9be79e1b1a0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:23.321072 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1419/2806] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:52:23 2022] POST /identity/v3/users => generated 301 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:52:23.325203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0d52e-c5f3-41a6-9658-422812667a57 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:23.336462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0d52e-c5f3-41a6-9658-422812667a57 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:23.342977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2ae2d-d9f2-4691-870f-f36a9d37aed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:23.343588 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2ae2d-d9f2-4691-870f-f36a9d37aed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:23.343873 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2ae2d-d9f2-4691-870f-f36a9d37aed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:23.344149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2ae2d-d9f2-4691-870f-f36a9d37aed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:23.344831 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2ae2d-d9f2-4691-870f-f36a9d37aed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:23.346429 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2ae2d-d9f2-4691-870f-f36a9d37aed8 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:23.356870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1388/2807] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:23.361104 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2e432-d547-4bb0-9980-e9be79e1b1a0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:23.371735 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2e432-d547-4bb0-9980-e9be79e1b1a0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:23.376407 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7132531-cc9a-4ed4-9562-9a049cf57ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:23.376993 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7132531-cc9a-4ed4-9562-9a049cf57ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:23.377273 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7132531-cc9a-4ed4-9562-9a049cf57ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:23.377540 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7132531-cc9a-4ed4-9562-9a049cf57ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:23.378172 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7132531-cc9a-4ed4-9562-9a049cf57ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:23.379763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7132531-cc9a-4ed4-9562-9a049cf57ab4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:23.390504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1420/2808] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:23.394804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2ae2d-d9f2-4691-870f-f36a9d37aed8 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:23.405585 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2ae2d-d9f2-4691-870f-f36a9d37aed8 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:23.410335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0aecbb-c696-4ca9-82a1-82a1e8020888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:23.410983 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aecbb-c696-4ca9-82a1-82a1e8020888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:23.411273 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aecbb-c696-4ca9-82a1-82a1e8020888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:23.411559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0aecbb-c696-4ca9-82a1-82a1e8020888 admin admin] PATH_INFO: `/v3/projects/95c671191b5644aa94fa3ef84acce281/users/2eda32845be44b9e9d438ea71cba66ab/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:23.426600 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aecbb-c696-4ca9-82a1-82a1e8020888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95c671191b5644aa94fa3ef84acce281, user_id=2eda32845be44b9e9d438ea71cba66ab, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:23.430575 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0aecbb-c696-4ca9-82a1-82a1e8020888 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:23.547578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1389/2809] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:23 2022] PUT /identity/v3/projects/95c671191b5644aa94fa3ef84acce281/users/2eda32845be44b9e9d438ea71cba66ab/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:23.559296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895012b-dc43-4193-b061-e68fa35b05cf None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:23.559590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895012b-dc43-4193-b061-e68fa35b05cf None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:23.562727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895012b-dc43-4193-b061-e68fa35b05cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:23.589490 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-c895012b-dc43-4193-b061-e68fa35b05cf None None] MFA Rules not processed for user `2eda32845be44b9e9d438ea71cba66ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:23.590705 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c895012b-dc43-4193-b061-e68fa35b05cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:23.658649 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1421/2810] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:23.665725 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0aecbb-c696-4ca9-82a1-82a1e8020888 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:23.727752 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0aecbb-c696-4ca9-82a1-82a1e8020888 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:23.732630 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13643ca-aacf-4308-bdc5-d1eb007440bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:23.733228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13643ca-aacf-4308-bdc5-d1eb007440bd admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:23.733504 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13643ca-aacf-4308-bdc5-d1eb007440bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:23.733829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13643ca-aacf-4308-bdc5-d1eb007440bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:23.734464 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13643ca-aacf-4308-bdc5-d1eb007440bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:23.736430 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13643ca-aacf-4308-bdc5-d1eb007440bd admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:23.857326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1390/2811] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:52:23 2022] POST /identity/v3/projects => generated 401 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:23.861387 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895012b-dc43-4193-b061-e68fa35b05cf None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:23.872818 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895012b-dc43-4193-b061-e68fa35b05cf None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:23.936840 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6b3c1-08c6-4675-a12e-039544913c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:23.937466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6b3c1-08c6-4675-a12e-039544913c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:23.938001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6b3c1-08c6-4675-a12e-039544913c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:23.938497 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6b3c1-08c6-4675-a12e-039544913c74 admin admin] PATH_INFO: `/v3/users` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:23.939146 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6b3c1-08c6-4675-a12e-039544913c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:23.941183 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6b3c1-08c6-4675-a12e-039544913c74 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:24.057470 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1422/2812] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:52:23 2022] POST /identity/v3/users => generated 307 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:24.061518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13643ca-aacf-4308-bdc5-d1eb007440bd admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:24.073489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13643ca-aacf-4308-bdc5-d1eb007440bd admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:24.078264 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e5c07-89c4-492f-912f-30109a622814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:24.078872 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e5c07-89c4-492f-912f-30109a622814 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:24.079152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e5c07-89c4-492f-912f-30109a622814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:24.079424 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e5c07-89c4-492f-912f-30109a622814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:24.080069 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e5c07-89c4-492f-912f-30109a622814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:24.081704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e5c07-89c4-492f-912f-30109a622814 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:24.093256 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1391/2813] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:24.097509 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6b3c1-08c6-4675-a12e-039544913c74 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:24.108123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6b3c1-08c6-4675-a12e-039544913c74 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:24.112710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1016-4576-452f-a0fd-53a36c3d2b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:24.113352 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1016-4576-452f-a0fd-53a36c3d2b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:24.113634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1016-4576-452f-a0fd-53a36c3d2b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:24.113906 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1016-4576-452f-a0fd-53a36c3d2b68 admin admin] PATH_INFO: `/v3/projects/b07784f7c1844db68621d4d4f0502e3a/users/e2572de3b2594ae98079c79885ab100e/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:24.129549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1016-4576-452f-a0fd-53a36c3d2b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07784f7c1844db68621d4d4f0502e3a, user_id=e2572de3b2594ae98079c79885ab100e, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:24.132795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1016-4576-452f-a0fd-53a36c3d2b68 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:24.212346 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1423/2814] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:24 2022] PUT /identity/v3/projects/b07784f7c1844db68621d4d4f0502e3a/users/e2572de3b2594ae98079c79885ab100e/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:24.216278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e5c07-89c4-492f-912f-30109a622814 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:24.227164 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e5c07-89c4-492f-912f-30109a622814 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:24.250398 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:24.297303 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7919e69-17cc-45ea-9418-01d0e5678df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:24.297937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7919e69-17cc-45ea-9418-01d0e5678df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:24.298220 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7919e69-17cc-45ea-9418-01d0e5678df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:24.298497 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7919e69-17cc-45ea-9418-01d0e5678df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:24.299262 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7919e69-17cc-45ea-9418-01d0e5678df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:24.301006 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7919e69-17cc-45ea-9418-01d0e5678df1 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:24.313492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1392/2815] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:24.319488 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1016-4576-452f-a0fd-53a36c3d2b68 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:24.325278 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:24.330900 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1016-4576-452f-a0fd-53a36c3d2b68 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:24.336860 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:24.343328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:24.343328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:24.343328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] PATH_INFO: `/v3/projects/b07784f7c1844db68621d4d4f0502e3a/users/e2572de3b2594ae98079c79885ab100e/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:24.343328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07784f7c1844db68621d4d4f0502e3a, user_id=e2572de3b2594ae98079c79885ab100e, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:24.343328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:24.356226 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1424/2816] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:24 2022] PUT /identity/v3/projects/b07784f7c1844db68621d4d4f0502e3a/users/e2572de3b2594ae98079c79885ab100e/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:24.423852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b3210-64c8-43d5-a4d7-03283e67c84d None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:24.424058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b3210-64c8-43d5-a4d7-03283e67c84d None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:24.424359 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b3210-64c8-43d5-a4d7-03283e67c84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:24.456811 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-779b3210-64c8-43d5-a4d7-03283e67c84d None None] MFA Rules not processed for user `e2572de3b2594ae98079c79885ab100e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:24.458048 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-779b3210-64c8-43d5-a4d7-03283e67c84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:24.529058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1393/2817] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:24.540918 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:24.542665 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:24.604401 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf301-7f69-42b2-909c-582c0abb0d6e admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:24.608118 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b519b5c-5209-429e-bba8-0cd806597495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:24.608780 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b519b5c-5209-429e-bba8-0cd806597495 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:24.609060 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b519b5c-5209-429e-bba8-0cd806597495 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:24.609334 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b519b5c-5209-429e-bba8-0cd806597495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:24.619642 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b519b5c-5209-429e-bba8-0cd806597495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:24.621108 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b519b5c-5209-429e-bba8-0cd806597495 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:24.630836 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1425/2818] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:24 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:52:24.832442 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b3210-64c8-43d5-a4d7-03283e67c84d None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:24.842730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b3210-64c8-43d5-a4d7-03283e67c84d None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:24.846349 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ebfa0-4809-4d4c-b9f1-c726e0608112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:24.846951 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebfa0-4809-4d4c-b9f1-c726e0608112 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:24.847235 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebfa0-4809-4d4c-b9f1-c726e0608112 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:24.847512 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebfa0-4809-4d4c-b9f1-c726e0608112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:24.904563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ebfa0-4809-4d4c-b9f1-c726e0608112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:24.905995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ebfa0-4809-4d4c-b9f1-c726e0608112 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:24.915173 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1394/2819] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:24 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:52:30.191078 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b519b5c-5209-429e-bba8-0cd806597495 service cinder] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:30.256822 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b519b5c-5209-429e-bba8-0cd806597495 service cinder] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:30.261356 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:30.261986 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:30.262270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:30.262548 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] PATH_INFO: `/v3/users/2eda32845be44b9e9d438ea71cba66ab` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:30.263675 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eda32845be44b9e9d438ea71cba66ab)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:30.265644 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:30.425006 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eda32845be44b9e9d438ea71cba66ab'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:30.425678 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eda32845be44b9e9d438ea71cba66ab'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:30.433582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eda32845be44b9e9d438ea71cba66ab'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:30.513425 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eda32845be44b9e9d438ea71cba66ab'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:30.513985 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eda32845be44b9e9d438ea71cba66ab'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:30.533046 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1426/2820] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:52:30 2022] DELETE /identity/v3/users/2eda32845be44b9e9d438ea71cba66ab => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:30.540284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637feb97-1ee1-4ceb-b9d2-fe3b3d685e3e None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:30.540284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637feb97-1ee1-4ceb-b9d2-fe3b3d685e3e None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:30.540284 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637feb97-1ee1-4ceb-b9d2-fe3b3d685e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:30.559520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-637feb97-1ee1-4ceb-b9d2-fe3b3d685e3e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:30.560770 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-637feb97-1ee1-4ceb-b9d2-fe3b3d685e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:30.601457 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=100 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:30.619790 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1395/2821] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:52:30.629565 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:30.631589 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:30.645129 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:30.690854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685eaa1-7a2e-4dd7-bfc5-c6576d54b11d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:30.695414 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:30.696013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:30.696332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:30.696608 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:30.705921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:30.707365 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:30.717035 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1427/2822] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:52:31.070547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637feb97-1ee1-4ceb-b9d2-fe3b3d685e3e None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:31.072028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-637feb97-1ee1-4ceb-b9d2-fe3b3d685e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:31.081102 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:31.082763 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:31.098491 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7540a6c-d111-46f2-a5e3-1f914cd9dbc9 service neutron] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:31.104419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:31.105236 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:31.105586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:31.105961 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] PATH_INFO: `/v3/projects/95c671191b5644aa94fa3ef84acce281` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:31.107269 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=95c671191b5644aa94fa3ef84acce281)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:31.109324 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:31.156067 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637feb97-1ee1-4ceb-b9d2-fe3b3d685e3e None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:31.161712 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:31.162342 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:31.162628 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:31.163159 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:31.164764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:31.176813 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:31.179094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:31.189453 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1396/2823] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:52:31.241064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95c671191b5644aa94fa3ef84acce281', 'request_id': 'req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.247583 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95c671191b5644aa94fa3ef84acce281', 'request_id': 'req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.329859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95c671191b5644aa94fa3ef84acce281', 'request_id': 'req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.342148 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] The token cache is being invalidate because project 95c671191b5644aa94fa3ef84acce281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:52:31.342441 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.343167 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.344764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1428/2824] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:52:31 2022] DELETE /identity/v3/projects/95c671191b5644aa94fa3ef84acce281 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:52:31.349335 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:31.360649 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f4400-bf91-47db-a716-945d41942b0b service placement] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:31.421559 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:31.422279 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:31.422569 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:31.422855 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] PATH_INFO: `/v3/users/e2572de3b2594ae98079c79885ab100e` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:31.429089 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2572de3b2594ae98079c79885ab100e)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:31.431211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:31.596255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2572de3b2594ae98079c79885ab100e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.596936 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2572de3b2594ae98079c79885ab100e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.604866 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2572de3b2594ae98079c79885ab100e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.605520 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2572de3b2594ae98079c79885ab100e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.695566 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-19458a25-c52c-4d1d-b083-da47deb652a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2572de3b2594ae98079c79885ab100e'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:31.706518 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:31.714436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1397/2825] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:52:31 2022] DELETE /identity/v3/users/e2572de3b2594ae98079c79885ab100e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:32.179963 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:32.182080 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:32.199181 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c199b-b94b-408b-ae2f-ba37e99bd61d admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:32.262247 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:32.262928 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:32.263168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:32.263426 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] PATH_INFO: `/v3/projects/b07784f7c1844db68621d4d4f0502e3a` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:32.264594 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07784f7c1844db68621d4d4f0502e3a)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:32.266524 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:32.384682 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:32.392450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07784f7c1844db68621d4d4f0502e3a', 'request_id': 'req-69739594-74cd-44eb-90cd-fa6af5d66296'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:32.399317 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07784f7c1844db68621d4d4f0502e3a', 'request_id': 'req-69739594-74cd-44eb-90cd-fa6af5d66296'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:32.462062 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07784f7c1844db68621d4d4f0502e3a', 'request_id': 'req-69739594-74cd-44eb-90cd-fa6af5d66296'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:32.473794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] The token cache is being invalidate because project b07784f7c1844db68621d4d4f0502e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:52:32.474199 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:32.474884 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-69739594-74cd-44eb-90cd-fa6af5d66296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:32.476436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1429/2826] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:52:32 2022] DELETE /identity/v3/projects/b07784f7c1844db68621d4d4f0502e3a => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:32.481336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e1133-a920-447f-b5cf-adfcee52cad1 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:32.481623 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e1133-a920-447f-b5cf-adfcee52cad1 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:32.481896 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e1133-a920-447f-b5cf-adfcee52cad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:32.502595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-674e1133-a920-447f-b5cf-adfcee52cad1 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:32.503806 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-674e1133-a920-447f-b5cf-adfcee52cad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:32.563943 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1398/2827] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:32.573481 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e227bf-3dbc-46bf-a82e-7040dd36ec0e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:32.573749 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e227bf-3dbc-46bf-a82e-7040dd36ec0e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:32.574026 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e227bf-3dbc-46bf-a82e-7040dd36ec0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:32.594755 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d9e227bf-3dbc-46bf-a82e-7040dd36ec0e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:32.595966 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d9e227bf-3dbc-46bf-a82e-7040dd36ec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:32.604802 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1430/2828] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:32.609299 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e1133-a920-447f-b5cf-adfcee52cad1 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:32.620426 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e1133-a920-447f-b5cf-adfcee52cad1 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:32.625086 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:32.625684 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:32.625964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:32.626236 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:32.626964 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:52:32.627265 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:32.629296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:32.640796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1399/2829] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:52:32.645348 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e227bf-3dbc-46bf-a82e-7040dd36ec0e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:32.655125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e227bf-3dbc-46bf-a82e-7040dd36ec0e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:32.661500 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ccfef-9c3f-4cb4-a249-a93a5cb67ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:32.661736 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ccfef-9c3f-4cb4-a249-a93a5cb67ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:32.664277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ccfef-9c3f-4cb4-a249-a93a5cb67ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:32.664277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ccfef-9c3f-4cb4-a249-a93a5cb67ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:32.664277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ccfef-9c3f-4cb4-a249-a93a5cb67ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:32.664898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ccfef-9c3f-4cb4-a249-a93a5cb67ec4 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:32.761205 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1431/2830] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:52:32 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:52:32.765276 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:32.775466 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae21bb-27e9-46b3-8d15-d05163436f91 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:32.832858 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfa842-84a9-4e49-be56-2dbefc868958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:32.833804 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfa842-84a9-4e49-be56-2dbefc868958 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:32.834091 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfa842-84a9-4e49-be56-2dbefc868958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:32.834371 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfa842-84a9-4e49-be56-2dbefc868958 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:32.835241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfa842-84a9-4e49-be56-2dbefc868958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:32.837217 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfa842-84a9-4e49-be56-2dbefc868958 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:32.925243 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1400/2831] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:52:32 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:32.930057 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ccfef-9c3f-4cb4-a249-a93a5cb67ec4 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:32.940842 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ccfef-9c3f-4cb4-a249-a93a5cb67ec4 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:32.945898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52b43d-e70c-4e37-a247-42cd194a7dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:32.946544 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52b43d-e70c-4e37-a247-42cd194a7dae admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:32.946824 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52b43d-e70c-4e37-a247-42cd194a7dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:32.947099 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52b43d-e70c-4e37-a247-42cd194a7dae admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:32.947782 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52b43d-e70c-4e37-a247-42cd194a7dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:32.949433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52b43d-e70c-4e37-a247-42cd194a7dae admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:32.961381 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1432/2832] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:32.965517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfa842-84a9-4e49-be56-2dbefc868958 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:32.977113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfa842-84a9-4e49-be56-2dbefc868958 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:32.981898 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d707e5d-a595-4bab-ac28-17ac5623d910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:32.982489 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d707e5d-a595-4bab-ac28-17ac5623d910 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:32.982769 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d707e5d-a595-4bab-ac28-17ac5623d910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:32.983037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d707e5d-a595-4bab-ac28-17ac5623d910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:32.983676 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d707e5d-a595-4bab-ac28-17ac5623d910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:32.985296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d707e5d-a595-4bab-ac28-17ac5623d910 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:32.996095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1401/2833] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:33.000337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52b43d-e70c-4e37-a247-42cd194a7dae admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:33.010185 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52b43d-e70c-4e37-a247-42cd194a7dae admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:33.014794 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7ce29-8a33-4914-9bbd-73371c69ede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:33.015479 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7ce29-8a33-4914-9bbd-73371c69ede7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:33.015710 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7ce29-8a33-4914-9bbd-73371c69ede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:33.015982 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7ce29-8a33-4914-9bbd-73371c69ede7 admin admin] PATH_INFO: `/v3/projects/abefe93469f74488b7108d3b52267817/users/b93b4ce7e9c94534aa596d5f4f24f9df/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:33.031902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7ce29-8a33-4914-9bbd-73371c69ede7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abefe93469f74488b7108d3b52267817, user_id=b93b4ce7e9c94534aa596d5f4f24f9df, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:33.034564 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7ce29-8a33-4914-9bbd-73371c69ede7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:33.137077 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1433/2834] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:32 2022] PUT /identity/v3/projects/abefe93469f74488b7108d3b52267817/users/b93b4ce7e9c94534aa596d5f4f24f9df/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:52:33.156789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586e356-7317-4192-a1af-735f433f8180 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:33.157078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586e356-7317-4192-a1af-735f433f8180 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:33.157349 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5586e356-7317-4192-a1af-735f433f8180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:33.185195 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-5586e356-7317-4192-a1af-735f433f8180 None None] MFA Rules not processed for user `b93b4ce7e9c94534aa596d5f4f24f9df`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:33.186390 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-5586e356-7317-4192-a1af-735f433f8180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:33.233870 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:33.251826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1402/2835] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:33.256512 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7ce29-8a33-4914-9bbd-73371c69ede7 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:33.332097 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7ce29-8a33-4914-9bbd-73371c69ede7 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:33.337560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e4c2e-6d83-44fb-a5f8-81a7a92a0224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:33.338161 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e4c2e-6d83-44fb-a5f8-81a7a92a0224 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:33.338442 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e4c2e-6d83-44fb-a5f8-81a7a92a0224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:33.338722 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e4c2e-6d83-44fb-a5f8-81a7a92a0224 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:33.339436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e4c2e-6d83-44fb-a5f8-81a7a92a0224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:33.341390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e4c2e-6d83-44fb-a5f8-81a7a92a0224 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:33.417519 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1434/2836] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:52:33 2022] POST /identity/v3/projects => generated 367 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:52:33.421619 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586e356-7317-4192-a1af-735f433f8180 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:33.435007 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5586e356-7317-4192-a1af-735f433f8180 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:33.492095 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdacbca-231a-4677-abaf-f0cba533a8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:33.492742 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdacbca-231a-4677-abaf-f0cba533a8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:33.493020 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdacbca-231a-4677-abaf-f0cba533a8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:33.493296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdacbca-231a-4677-abaf-f0cba533a8c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:33.493918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdacbca-231a-4677-abaf-f0cba533a8c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:33.495829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdacbca-231a-4677-abaf-f0cba533a8c9 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:33.667255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1403/2837] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:52:33 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:33.671328 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e4c2e-6d83-44fb-a5f8-81a7a92a0224 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:33.682692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e4c2e-6d83-44fb-a5f8-81a7a92a0224 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:33.687436 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669b601-12e6-4276-a523-e3a52f239846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:33.688038 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669b601-12e6-4276-a523-e3a52f239846 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:33.688351 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669b601-12e6-4276-a523-e3a52f239846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:33.688631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669b601-12e6-4276-a523-e3a52f239846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:33.689306 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669b601-12e6-4276-a523-e3a52f239846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:33.690906 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669b601-12e6-4276-a523-e3a52f239846 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:33.703013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1435/2838] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:52:33.707389 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdacbca-231a-4677-abaf-f0cba533a8c9 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:33.719657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdacbca-231a-4677-abaf-f0cba533a8c9 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:33.725362 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f86b0-bde1-4cff-848c-a75f3dc04354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:33.726037 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f86b0-bde1-4cff-848c-a75f3dc04354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:33.726321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f86b0-bde1-4cff-848c-a75f3dc04354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:33.726595 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f86b0-bde1-4cff-848c-a75f3dc04354 admin admin] PATH_INFO: `/v3/projects/2bb9d5e1b970480bb291a88c01126939/users/7ae99aa0ea144f2dae1b5a3e1567848f/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:33.744540 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f86b0-bde1-4cff-848c-a75f3dc04354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb9d5e1b970480bb291a88c01126939, user_id=7ae99aa0ea144f2dae1b5a3e1567848f, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:33.747233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f86b0-bde1-4cff-848c-a75f3dc04354 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:33.949211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1404/2839] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:33 2022] PUT /identity/v3/projects/2bb9d5e1b970480bb291a88c01126939/users/7ae99aa0ea144f2dae1b5a3e1567848f/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:52:34.015243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669b601-12e6-4276-a523-e3a52f239846 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:34.015243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669b601-12e6-4276-a523-e3a52f239846 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:34.015243 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:34.021020 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f883b-9845-42e7-9200-86f5ed728171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:34.021648 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f883b-9845-42e7-9200-86f5ed728171 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:34.021934 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f883b-9845-42e7-9200-86f5ed728171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:34.022210 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f883b-9845-42e7-9200-86f5ed728171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:34.022882 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f883b-9845-42e7-9200-86f5ed728171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:34.024502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f883b-9845-42e7-9200-86f5ed728171 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:34.034879 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1436/2840] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:52:34.039024 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f86b0-bde1-4cff-848c-a75f3dc04354 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:34.049939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f86b0-bde1-4cff-848c-a75f3dc04354 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:34.055416 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:34.056059 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:34.056370 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:34.056645 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] PATH_INFO: `/v3/projects/2bb9d5e1b970480bb291a88c01126939/users/7ae99aa0ea144f2dae1b5a3e1567848f/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:34.058571 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb9d5e1b970480bb291a88c01126939, user_id=7ae99aa0ea144f2dae1b5a3e1567848f, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:34.061285 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:34.073366 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1405/2841] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:34 2022] PUT /identity/v3/projects/2bb9d5e1b970480bb291a88c01126939/users/7ae99aa0ea144f2dae1b5a3e1567848f/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:34.084449 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c87dc-2739-48a0-8826-c1348e12c900 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:34.084995 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c87dc-2739-48a0-8826-c1348e12c900 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:34.085452 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c87dc-2739-48a0-8826-c1348e12c900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:34.115196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-ca0c87dc-2739-48a0-8826-c1348e12c900 None None] MFA Rules not processed for user `7ae99aa0ea144f2dae1b5a3e1567848f`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:34.118455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-ca0c87dc-2739-48a0-8826-c1348e12c900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:34.185560 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1437/2842] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:34.197257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:34.198977 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:34.257437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23d2e4-1543-4db2-958c-1732c9aa1bcb admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:34.261087 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cd7d9-e903-43c1-bd08-cf3692caba36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:34.261694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd7d9-e903-43c1-bd08-cf3692caba36 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:34.261970 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd7d9-e903-43c1-bd08-cf3692caba36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:34.262242 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd7d9-e903-43c1-bd08-cf3692caba36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:34.271321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cd7d9-e903-43c1-bd08-cf3692caba36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:34.272796 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884cd7d9-e903-43c1-bd08-cf3692caba36 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:34.282196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1406/2843] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:52:34.523314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c87dc-2739-48a0-8826-c1348e12c900 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:34.533061 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c87dc-2739-48a0-8826-c1348e12c900 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:34.537261 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997e724-6088-416a-aab7-3e219707cf3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:34.537859 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997e724-6088-416a-aab7-3e219707cf3d service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:34.538141 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997e724-6088-416a-aab7-3e219707cf3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:34.538416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997e724-6088-416a-aab7-3e219707cf3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:34.595732 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997e724-6088-416a-aab7-3e219707cf3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:34.597270 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a997e724-6088-416a-aab7-3e219707cf3d service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:34.607555 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1438/2844] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:52:41.112683 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cd7d9-e903-43c1-bd08-cf3692caba36 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:41.157448 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:41.174705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cd7d9-e903-43c1-bd08-cf3692caba36 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:41.180388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:41.181070 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:41.181564 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:41.182119 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] PATH_INFO: `/v3/users/b93b4ce7e9c94534aa596d5f4f24f9df` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:41.183789 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b93b4ce7e9c94534aa596d5f4f24f9df)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:41.185825 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:41.322116 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b93b4ce7e9c94534aa596d5f4f24f9df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:41.322776 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b93b4ce7e9c94534aa596d5f4f24f9df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:41.329805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b93b4ce7e9c94534aa596d5f4f24f9df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:41.330436 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b93b4ce7e9c94534aa596d5f4f24f9df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:41.370367 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b93b4ce7e9c94534aa596d5f4f24f9df'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:41.388233 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1407/2845] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:52:41 2022] DELETE /identity/v3/users/b93b4ce7e9c94534aa596d5f4f24f9df => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:41.392885 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a747-9a5e-4b80-8eb5-b2ec0bdb2b99 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:41.393379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a747-9a5e-4b80-8eb5-b2ec0bdb2b99 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:41.393511 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a747-9a5e-4b80-8eb5-b2ec0bdb2b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:41.413476 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-5a80a747-9a5e-4b80-8eb5-b2ec0bdb2b99 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:41.418271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5a80a747-9a5e-4b80-8eb5-b2ec0bdb2b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:41.466296 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=100 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:41.476728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1439/2846] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:41.486516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:41.487934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:41.546874 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6dd44b-65f0-4e4f-9226-ad069c166b84 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:41.550578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc19f2-853a-4201-9495-d15e0ea05706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:41.551171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc19f2-853a-4201-9495-d15e0ea05706 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:41.551509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc19f2-853a-4201-9495-d15e0ea05706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:41.551805 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc19f2-853a-4201-9495-d15e0ea05706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:41.560939 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc19f2-853a-4201-9495-d15e0ea05706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:41.562563 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc19f2-853a-4201-9495-d15e0ea05706 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:41.573178 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1408/2847] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 13:52:41.983689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80a747-9a5e-4b80-8eb5-b2ec0bdb2b99 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:41.985202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-5a80a747-9a5e-4b80-8eb5-b2ec0bdb2b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:42.000109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80a747-9a5e-4b80-8eb5-b2ec0bdb2b99 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:42.005868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:42.009255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:42.009255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:42.009255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] PATH_INFO: `/v3/projects/abefe93469f74488b7108d3b52267817` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:42.009255 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abefe93469f74488b7108d3b52267817)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:42.010191 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:42.114972 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abefe93469f74488b7108d3b52267817', 'request_id': 'req-66dcdba1-58ab-4616-b6ad-bd6fb1848241'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.120864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abefe93469f74488b7108d3b52267817', 'request_id': 'req-66dcdba1-58ab-4616-b6ad-bd6fb1848241'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.211395 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abefe93469f74488b7108d3b52267817', 'request_id': 'req-66dcdba1-58ab-4616-b6ad-bd6fb1848241'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.223983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] The token cache is being invalidate because project abefe93469f74488b7108d3b52267817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:52:42.223983 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.224707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.226237 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1440/2848] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:52:41 2022] DELETE /identity/v3/projects/abefe93469f74488b7108d3b52267817 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:42.230189 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc19f2-853a-4201-9495-d15e0ea05706 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:42.241519 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc19f2-853a-4201-9495-d15e0ea05706 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:42.299730 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:42.300388 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:42.300692 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:42.300950 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] PATH_INFO: `/v3/users/7ae99aa0ea144f2dae1b5a3e1567848f` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:42.303228 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae99aa0ea144f2dae1b5a3e1567848f)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:42.305198 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:42.310551 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:42.567711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae99aa0ea144f2dae1b5a3e1567848f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.568401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae99aa0ea144f2dae1b5a3e1567848f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.575516 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae99aa0ea144f2dae1b5a3e1567848f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.576172 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae99aa0ea144f2dae1b5a3e1567848f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.673423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-ca94baa7-65bd-4333-b2d3-405ebc5e7a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae99aa0ea144f2dae1b5a3e1567848f'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:42.695678 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1409/2849] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:52:42 2022] DELETE /identity/v3/users/7ae99aa0ea144f2dae1b5a3e1567848f => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:43.241178 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:43.242662 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:43.256315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcdba1-58ab-4616-b6ad-bd6fb1848241 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:43.322783 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:43.323450 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:43.323725 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:43.324008 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] PATH_INFO: `/v3/projects/2bb9d5e1b970480bb291a88c01126939` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:43.325207 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb9d5e1b970480bb291a88c01126939)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:43.327119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:43.464433 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb9d5e1b970480bb291a88c01126939', 'request_id': 'req-921bc810-aca7-4737-8aa1-8976779bd1c1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:43.469976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb9d5e1b970480bb291a88c01126939', 'request_id': 'req-921bc810-aca7-4737-8aa1-8976779bd1c1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:43.524883 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb9d5e1b970480bb291a88c01126939', 'request_id': 'req-921bc810-aca7-4737-8aa1-8976779bd1c1'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:43.536707 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] The token cache is being invalidate because project 2bb9d5e1b970480bb291a88c01126939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:52:43.536991 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:43.537620 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-921bc810-aca7-4737-8aa1-8976779bd1c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:43.539106 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1441/2850] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:52:43 2022] DELETE /identity/v3/projects/2bb9d5e1b970480bb291a88c01126939 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:43.544667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9524216-5828-4a22-bc32-80f3826cdf43 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:43.545144 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9524216-5828-4a22-bc32-80f3826cdf43 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:43.545241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9524216-5828-4a22-bc32-80f3826cdf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:43.565021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d9524216-5828-4a22-bc32-80f3826cdf43 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:43.566244 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d9524216-5828-4a22-bc32-80f3826cdf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:43.627016 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1410/2851] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:43.635741 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f49ad-a985-41ba-9c2e-a7f5763b659e None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:43.636013 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f49ad-a985-41ba-9c2e-a7f5763b659e None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:43.636316 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f49ad-a985-41ba-9c2e-a7f5763b659e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:43.657874 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8e6f49ad-a985-41ba-9c2e-a7f5763b659e None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:43.659087 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8e6f49ad-a985-41ba-9c2e-a7f5763b659e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:43.667514 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1442/2852] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:43.672204 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9524216-5828-4a22-bc32-80f3826cdf43 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:43.683826 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9524216-5828-4a22-bc32-80f3826cdf43 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:43.688544 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:43.689150 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:43.689434 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:43.689711 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:43.690363 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:52:43.690668 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:43.693292 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:43.705112 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1411/2853] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:52:43.710219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f49ad-a985-41ba-9c2e-a7f5763b659e None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:43.715921 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:43.719771 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f49ad-a985-41ba-9c2e-a7f5763b659e None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:43.724969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd9cc2-e6e4-42bc-9dfa-90166c4449f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:43.726634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd9cc2-e6e4-42bc-9dfa-90166c4449f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:43.726634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd9cc2-e6e4-42bc-9dfa-90166c4449f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:43.726634 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd9cc2-e6e4-42bc-9dfa-90166c4449f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:43.727495 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd9cc2-e6e4-42bc-9dfa-90166c4449f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:43.728694 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd9cc2-e6e4-42bc-9dfa-90166c4449f1 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:43.846135 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1443/2854] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:52:43 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:43.850843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:43.861351 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b0fa4-0102-42ec-beb0-a5a018277283 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:43.918521 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578f4e2-fad9-4026-9f08-e18499d5b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:43.919120 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f4e2-fad9-4026-9f08-e18499d5b5bf admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:43.919401 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f4e2-fad9-4026-9f08-e18499d5b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:43.919671 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578f4e2-fad9-4026-9f08-e18499d5b5bf admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:43.920360 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578f4e2-fad9-4026-9f08-e18499d5b5bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:43.922326 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578f4e2-fad9-4026-9f08-e18499d5b5bf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:43.994383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1412/2855] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:52:43 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 13:52:43.999128 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd9cc2-e6e4-42bc-9dfa-90166c4449f1 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.016614 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd9cc2-e6e4-42bc-9dfa-90166c4449f1 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.022633 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d0202-3c59-4d98-b7bd-3e1d611ba677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.022784 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0202-3c59-4d98-b7bd-3e1d611ba677 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.023121 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0202-3c59-4d98-b7bd-3e1d611ba677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.023413 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0202-3c59-4d98-b7bd-3e1d611ba677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.024085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d0202-3c59-4d98-b7bd-3e1d611ba677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.025728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d0202-3c59-4d98-b7bd-3e1d611ba677 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.036374 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1444/2856] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:52:44.040664 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578f4e2-fad9-4026-9f08-e18499d5b5bf admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.045960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:44.052055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578f4e2-fad9-4026-9f08-e18499d5b5bf admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.057345 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a4f22-26b5-4781-878c-392a95831cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.057955 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a4f22-26b5-4781-878c-392a95831cac admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.058241 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a4f22-26b5-4781-878c-392a95831cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.058517 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a4f22-26b5-4781-878c-392a95831cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.059206 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a4f22-26b5-4781-878c-392a95831cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.060934 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a4f22-26b5-4781-878c-392a95831cac admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.072941 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1413/2857] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:52:44.077122 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d0202-3c59-4d98-b7bd-3e1d611ba677 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.087518 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d0202-3c59-4d98-b7bd-3e1d611ba677 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.092248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9e2a9-9b35-4a4b-851b-b8254a9188d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.092898 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e2a9-9b35-4a4b-851b-b8254a9188d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.093179 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e2a9-9b35-4a4b-851b-b8254a9188d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.093453 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e2a9-9b35-4a4b-851b-b8254a9188d9 admin admin] PATH_INFO: `/v3/projects/23ee218687ab4bf095691e7e866072e4/users/dc6c4a189ffb4bcda20c8b79e6546299/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.108494 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9e2a9-9b35-4a4b-851b-b8254a9188d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ee218687ab4bf095691e7e866072e4, user_id=dc6c4a189ffb4bcda20c8b79e6546299, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.111175 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9e2a9-9b35-4a4b-851b-b8254a9188d9 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.182525 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1445/2858] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:44 2022] PUT /identity/v3/projects/23ee218687ab4bf095691e7e866072e4/users/dc6c4a189ffb4bcda20c8b79e6546299/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:44.195217 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1129752-a94d-45eb-82f3-c6853eea6d3c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.195510 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1129752-a94d-45eb-82f3-c6853eea6d3c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.195792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1129752-a94d-45eb-82f3-c6853eea6d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.225546 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-d1129752-a94d-45eb-82f3-c6853eea6d3c None None] MFA Rules not processed for user `dc6c4a189ffb4bcda20c8b79e6546299`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:44.226792 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d1129752-a94d-45eb-82f3-c6853eea6d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:44.292152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1414/2859] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:44.297523 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9e2a9-9b35-4a4b-851b-b8254a9188d9 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.361256 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9e2a9-9b35-4a4b-851b-b8254a9188d9 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.366619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7376de-3450-40df-a50d-82ecd9fa2406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.367218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7376de-3450-40df-a50d-82ecd9fa2406 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.367502 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7376de-3450-40df-a50d-82ecd9fa2406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.367795 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7376de-3450-40df-a50d-82ecd9fa2406 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.368435 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7376de-3450-40df-a50d-82ecd9fa2406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.370329 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7376de-3450-40df-a50d-82ecd9fa2406 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.464520 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1446/2860] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:52:44 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:44.468756 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1129752-a94d-45eb-82f3-c6853eea6d3c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.480749 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1129752-a94d-45eb-82f3-c6853eea6d3c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.538113 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bafa8-8b1d-46ac-9b62-83af0acce4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.538716 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bafa8-8b1d-46ac-9b62-83af0acce4de admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.538991 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bafa8-8b1d-46ac-9b62-83af0acce4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.539258 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bafa8-8b1d-46ac-9b62-83af0acce4de admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.539892 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bafa8-8b1d-46ac-9b62-83af0acce4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.541833 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bafa8-8b1d-46ac-9b62-83af0acce4de admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.651883 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1415/2861] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:52:44 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:44.655623 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7376de-3450-40df-a50d-82ecd9fa2406 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.666940 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7376de-3450-40df-a50d-82ecd9fa2406 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.671496 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f92d3c-5d00-45c4-9518-40e8c79819fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.672145 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f92d3c-5d00-45c4-9518-40e8c79819fe admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.672455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f92d3c-5d00-45c4-9518-40e8c79819fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.672728 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f92d3c-5d00-45c4-9518-40e8c79819fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.673372 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f92d3c-5d00-45c4-9518-40e8c79819fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.674959 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f92d3c-5d00-45c4-9518-40e8c79819fe admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.686403 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1447/2862] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:44.690630 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bafa8-8b1d-46ac-9b62-83af0acce4de admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.701238 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bafa8-8b1d-46ac-9b62-83af0acce4de admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.705944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b39c40-6205-4b35-8076-e4e5d62db925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.706592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b39c40-6205-4b35-8076-e4e5d62db925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.706878 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b39c40-6205-4b35-8076-e4e5d62db925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.707157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b39c40-6205-4b35-8076-e4e5d62db925 admin admin] PATH_INFO: `/v3/projects/464333a9e19a4aba97dc5900bc1f2437/users/a53cb35172be48c6850747d8410dac24/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.722352 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b39c40-6205-4b35-8076-e4e5d62db925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=464333a9e19a4aba97dc5900bc1f2437, user_id=a53cb35172be48c6850747d8410dac24, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.725652 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b39c40-6205-4b35-8076-e4e5d62db925 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.794324 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1416/2863] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:44 2022] PUT /identity/v3/projects/464333a9e19a4aba97dc5900bc1f2437/users/a53cb35172be48c6850747d8410dac24/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:44.798321 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f92d3c-5d00-45c4-9518-40e8c79819fe admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.808631 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f92d3c-5d00-45c4-9518-40e8c79819fe admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.866770 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3bb358-1aef-455b-a060-aef365630fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.867408 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb358-1aef-455b-a060-aef365630fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.867692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb358-1aef-455b-a060-aef365630fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.867969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb358-1aef-455b-a060-aef365630fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.868651 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bb358-1aef-455b-a060-aef365630fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.870257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3bb358-1aef-455b-a060-aef365630fc8 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.880288 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1448/2864] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:44.884451 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b39c40-6205-4b35-8076-e4e5d62db925 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:44.895547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b39c40-6205-4b35-8076-e4e5d62db925 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:44.900829 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:44.901460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.901744 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.902076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] PATH_INFO: `/v3/projects/464333a9e19a4aba97dc5900bc1f2437/users/a53cb35172be48c6850747d8410dac24/roles/b1b101411f4d4453a0eb2b378eeab684` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.903972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=464333a9e19a4aba97dc5900bc1f2437, user_id=a53cb35172be48c6850747d8410dac24, role_id=b1b101411f4d4453a0eb2b378eeab684)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:44.906658 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:44.920387 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1417/2865] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:44 2022] PUT /identity/v3/projects/464333a9e19a4aba97dc5900bc1f2437/users/a53cb35172be48c6850747d8410dac24/roles/b1b101411f4d4453a0eb2b378eeab684 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:44.931689 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b0220-43e6-451e-8224-ab7d0e9ad779 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:44.932085 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b0220-43e6-451e-8224-ab7d0e9ad779 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:44.932399 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b0220-43e6-451e-8224-ab7d0e9ad779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:44.961484 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-d65b0220-43e6-451e-8224-ab7d0e9ad779 None None] MFA Rules not processed for user `a53cb35172be48c6850747d8410dac24`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:44.962692 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d65b0220-43e6-451e-8224-ab7d0e9ad779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:44.967688 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:45.030951 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1449/2866] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:45.041944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:45.043601 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:45.079357 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:45.102486 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e2e1-57fd-4ae0-83c1-875abdfe2e92 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:45.106290 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f972a-5574-4c7c-aeaf-bb497bcc5b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:45.106889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f972a-5574-4c7c-aeaf-bb497bcc5b56 service cinder] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:45.107171 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f972a-5574-4c7c-aeaf-bb497bcc5b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:45.107447 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f972a-5574-4c7c-aeaf-bb497bcc5b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:45.164056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f972a-5574-4c7c-aeaf-bb497bcc5b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:45.165462 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f972a-5574-4c7c-aeaf-bb497bcc5b56 service cinder] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:45.174500 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1418/2867] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:52:50.459197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b0220-43e6-451e-8224-ab7d0e9ad779 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:50.460663 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-d65b0220-43e6-451e-8224-ab7d0e9ad779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:50.473612 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b0220-43e6-451e-8224-ab7d0e9ad779 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:50.477325 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe67451-ad89-416f-ad07-44622c324ec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:50.477919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe67451-ad89-416f-ad07-44622c324ec7 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:50.478197 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe67451-ad89-416f-ad07-44622c324ec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:50.478467 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe67451-ad89-416f-ad07-44622c324ec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:50.487391 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe67451-ad89-416f-ad07-44622c324ec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:50.488880 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe67451-ad89-416f-ad07-44622c324ec7 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:50.498289 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1450/2868] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:52:55.187465 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=33 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:56.498428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f972a-5574-4c7c-aeaf-bb497bcc5b56 service cinder] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:56.573267 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f972a-5574-4c7c-aeaf-bb497bcc5b56 service cinder] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:56.578138 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:56.578778 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:56.579063 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:56.579341 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] PATH_INFO: `/v3/users/dc6c4a189ffb4bcda20c8b79e6546299` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:56.580685 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6c4a189ffb4bcda20c8b79e6546299)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:56.583353 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:56.756055 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6c4a189ffb4bcda20c8b79e6546299'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:56.756780 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6c4a189ffb4bcda20c8b79e6546299'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:56.763657 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6c4a189ffb4bcda20c8b79e6546299'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:56.764330 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6c4a189ffb4bcda20c8b79e6546299'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:56.850612 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6c4a189ffb4bcda20c8b79e6546299'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:56.868959 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1419/2869] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:52:56 2022] DELETE /identity/v3/users/dc6c4a189ffb4bcda20c8b79e6546299 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:56.873746 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe49c3-10fd-4c8c-bd6d-90263cbabbfa None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:56.874277 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe49c3-10fd-4c8c-bd6d-90263cbabbfa None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:56.874390 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe49c3-10fd-4c8c-bd6d-90263cbabbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:56.893466 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-dafe49c3-10fd-4c8c-bd6d-90263cbabbfa None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:56.895196 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dafe49c3-10fd-4c8c-bd6d-90263cbabbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:56.954240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1451/2870] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:56.964737 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:56.966170 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:57.031534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a795914-7246-48bd-ad02-697e73b62396 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:57.035237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e27ac-a4fc-42b5-9606-750ada0f0fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:57.035974 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e27ac-a4fc-42b5-9606-750ada0f0fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:57.036296 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e27ac-a4fc-42b5-9606-750ada0f0fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:57.036578 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e27ac-a4fc-42b5-9606-750ada0f0fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:57.045938 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e27ac-a4fc-42b5-9606-750ada0f0fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:57.047393 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e27ac-a4fc-42b5-9606-750ada0f0fc6 service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:57.064532 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1420/2871] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:52:57.498309 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe49c3-10fd-4c8c-bd6d-90263cbabbfa None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:57.499971 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dafe49c3-10fd-4c8c-bd6d-90263cbabbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:57.516419 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe49c3-10fd-4c8c-bd6d-90263cbabbfa None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:57.519109 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:57.520405 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:57.520891 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:57.521251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] PATH_INFO: `/v3/projects/23ee218687ab4bf095691e7e866072e4` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:57.522618 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ee218687ab4bf095691e7e866072e4)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:57.524737 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:57.646549 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ee218687ab4bf095691e7e866072e4', 'request_id': 'req-2430a046-362e-4c1f-9d5f-5887dac71868'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:57.652533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ee218687ab4bf095691e7e866072e4', 'request_id': 'req-2430a046-362e-4c1f-9d5f-5887dac71868'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:57.742055 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ee218687ab4bf095691e7e866072e4', 'request_id': 'req-2430a046-362e-4c1f-9d5f-5887dac71868'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:57.759285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] The token cache is being invalidate because project 23ee218687ab4bf095691e7e866072e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:52:57.759582 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:57.760304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:57.761821 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1452/2872] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:52:57 2022] DELETE /identity/v3/projects/23ee218687ab4bf095691e7e866072e4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:57.766156 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e27ac-a4fc-42b5-9606-750ada0f0fc6 service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:57.776814 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e27ac-a4fc-42b5-9606-750ada0f0fc6 service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:57.795196 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:57.840460 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:57.841321 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:57.841598 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:57.841871 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] PATH_INFO: `/v3/users/a53cb35172be48c6850747d8410dac24` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:57.843094 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a53cb35172be48c6850747d8410dac24)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:57.845058 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:58.003587 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a53cb35172be48c6850747d8410dac24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.004963 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a53cb35172be48c6850747d8410dac24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.014673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a53cb35172be48c6850747d8410dac24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.015374 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a53cb35172be48c6850747d8410dac24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.078760 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-54f7f630-14ee-412e-950d-02ae842752b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a53cb35172be48c6850747d8410dac24'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.099565 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1421/2873] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:52:57 2022] DELETE /identity/v3/users/a53cb35172be48c6850747d8410dac24 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:58.520584 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:58.522082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:58.536101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430a046-362e-4c1f-9d5f-5887dac71868 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:58.546394 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:58.595684 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:58.596345 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:58.596629 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:58.596902 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] PATH_INFO: `/v3/projects/464333a9e19a4aba97dc5900bc1f2437` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:58.598656 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] RBAC: Authorizing `identity:delete_project(project_id=464333a9e19a4aba97dc5900bc1f2437)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:58.600611 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:58.718416 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '464333a9e19a4aba97dc5900bc1f2437', 'request_id': 'req-7221038d-6024-44ca-a8f0-2df2592ac37a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.725332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '464333a9e19a4aba97dc5900bc1f2437', 'request_id': 'req-7221038d-6024-44ca-a8f0-2df2592ac37a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.814869 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '464333a9e19a4aba97dc5900bc1f2437', 'request_id': 'req-7221038d-6024-44ca-a8f0-2df2592ac37a'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.827970 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] The token cache is being invalidate because project 464333a9e19a4aba97dc5900bc1f2437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:52:58.828315 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.829066 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-7221038d-6024-44ca-a8f0-2df2592ac37a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:52:58.830687 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1453/2874] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:52:58 2022] DELETE /identity/v3/projects/464333a9e19a4aba97dc5900bc1f2437 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:52:58.837291 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea3099-4a0e-46da-bd47-ac2dc0443e4c None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:58.837788 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea3099-4a0e-46da-bd47-ac2dc0443e4c None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:58.837918 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea3099-4a0e-46da-bd47-ac2dc0443e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:58.860795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-fcea3099-4a0e-46da-bd47-ac2dc0443e4c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:58.862072 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-fcea3099-4a0e-46da-bd47-ac2dc0443e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:58.922379 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1422/2875] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:52:58.932166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035e22a-7cbe-4708-aa06-7ec61d303220 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:58.932482 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035e22a-7cbe-4708-aa06-7ec61d303220 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:58.932759 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035e22a-7cbe-4708-aa06-7ec61d303220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:58.952952 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-8035e22a-7cbe-4708-aa06-7ec61d303220 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:58.954166 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-8035e22a-7cbe-4708-aa06-7ec61d303220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:58.967723 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1454/2876] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:58.970888 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea3099-4a0e-46da-bd47-ac2dc0443e4c None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:58.981818 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea3099-4a0e-46da-bd47-ac2dc0443e4c None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:58.986726 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:58.987331 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:58.987615 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:58.987889 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:58.988625 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:52:58.988937 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:58.990592 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:59.002149 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1423/2877] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:52:59.007027 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035e22a-7cbe-4708-aa06-7ec61d303220 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:59.016574 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035e22a-7cbe-4708-aa06-7ec61d303220 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:59.022271 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcfe88-a546-481d-9ee4-ca9405d97da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:59.022868 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcfe88-a546-481d-9ee4-ca9405d97da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:59.023257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcfe88-a546-481d-9ee4-ca9405d97da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:59.023533 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcfe88-a546-481d-9ee4-ca9405d97da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:59.024159 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bcfe88-a546-481d-9ee4-ca9405d97da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:59.026123 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bcfe88-a546-481d-9ee4-ca9405d97da0 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:59.155468 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1455/2878] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:52:59 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:59.159795 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:59.171021 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace2e82-6700-4c5e-b4e9-7cae293222cf admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:59.197694 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:59.228449 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb6552-283b-4dba-a08e-4b4a001c3021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:59.229054 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb6552-283b-4dba-a08e-4b4a001c3021 admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:59.229340 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb6552-283b-4dba-a08e-4b4a001c3021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:59.229610 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb6552-283b-4dba-a08e-4b4a001c3021 admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:59.230232 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb6552-283b-4dba-a08e-4b4a001c3021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:59.232136 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb6552-283b-4dba-a08e-4b4a001c3021 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:59.337224 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1424/2879] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:52:59 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:52:59.341400 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcfe88-a546-481d-9ee4-ca9405d97da0 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:59.352251 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcfe88-a546-481d-9ee4-ca9405d97da0 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:59.357024 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b36153-fa34-41eb-a6c9-c178698b080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:59.357619 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b36153-fa34-41eb-a6c9-c178698b080c admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:59.357942 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b36153-fa34-41eb-a6c9-c178698b080c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:59.358219 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b36153-fa34-41eb-a6c9-c178698b080c admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:59.358867 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b36153-fa34-41eb-a6c9-c178698b080c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:59.360504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b36153-fa34-41eb-a6c9-c178698b080c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:59.371621 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1456/2880] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:59.375840 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb6552-283b-4dba-a08e-4b4a001c3021 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:59.392334 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb6552-283b-4dba-a08e-4b4a001c3021 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:59.399045 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46503459-6752-421a-a639-ed5bc7f3439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:59.399659 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46503459-6752-421a-a639-ed5bc7f3439f admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:59.399944 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46503459-6752-421a-a639-ed5bc7f3439f admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:59.400251 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46503459-6752-421a-a639-ed5bc7f3439f admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:59.400962 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46503459-6752-421a-a639-ed5bc7f3439f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:59.402577 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46503459-6752-421a-a639-ed5bc7f3439f admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:59.414479 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1425/2881] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:52:59.418545 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b36153-fa34-41eb-a6c9-c178698b080c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:59.429149 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b36153-fa34-41eb-a6c9-c178698b080c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:59.433945 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:59.434600 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:59.434890 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:59.435168 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] PATH_INFO: `/v3/projects/cd45ad4a80ab4bf0a4a85e5271b13f8d/users/853cb0f3d5c147fbb8a87ae0a0a320f1/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:59.450976 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd45ad4a80ab4bf0a4a85e5271b13f8d, user_id=853cb0f3d5c147fbb8a87ae0a0a320f1, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:59.453679 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:59.537440 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1457/2882] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:52:59 2022] PUT /identity/v3/projects/cd45ad4a80ab4bf0a4a85e5271b13f8d/users/853cb0f3d5c147fbb8a87ae0a0a320f1/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:52:59.548679 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834f9b-310e-4029-a540-f2099f1ee972 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:59.549003 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834f9b-310e-4029-a540-f2099f1ee972 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:59.549283 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02834f9b-310e-4029-a540-f2099f1ee972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:59.578336 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-02834f9b-310e-4029-a540-f2099f1ee972 None None] MFA Rules not processed for user `853cb0f3d5c147fbb8a87ae0a0a320f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:52:59.579550 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-02834f9b-310e-4029-a540-f2099f1ee972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:59.644543 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1426/2883] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:52:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:52:59.655973 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:52:59.657764 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:52:59.701677 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:52:59.715332 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45509e-16bf-4e23-b7dc-512641a9463c admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:52:59.730919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bcf9f-dbdd-47f4-a6af-3f423794d24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:52:59.730919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bcf9f-dbdd-47f4-a6af-3f423794d24a service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:52:59.730919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bcf9f-dbdd-47f4-a6af-3f423794d24a service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:52:59.730919 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bcf9f-dbdd-47f4-a6af-3f423794d24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:52:59.731367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bcf9f-dbdd-47f4-a6af-3f423794d24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:52:59.731367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bcf9f-dbdd-47f4-a6af-3f423794d24a service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:52:59.739568 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1458/2884] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:52:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:53:09.641667 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:53:09.843202 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=8 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:53:09.959278 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02834f9b-310e-4029-a540-f2099f1ee972 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:10.020154 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02834f9b-310e-4029-a540-f2099f1ee972 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:10.025864 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:10.026492 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:10.026777 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:10.027056 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] PATH_INFO: `/v3/users/853cb0f3d5c147fbb8a87ae0a0a320f1` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:10.028259 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=853cb0f3d5c147fbb8a87ae0a0a320f1)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:10.030157 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:10.333703 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853cb0f3d5c147fbb8a87ae0a0a320f1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:10.334298 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853cb0f3d5c147fbb8a87ae0a0a320f1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:10.341248 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853cb0f3d5c147fbb8a87ae0a0a320f1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:10.341885 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853cb0f3d5c147fbb8a87ae0a0a320f1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:10.433076 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853cb0f3d5c147fbb8a87ae0a0a320f1'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:10.451211 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1427/2885] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:53:09 2022] DELETE /identity/v3/users/853cb0f3d5c147fbb8a87ae0a0a320f1 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:53:10.455751 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefcaec-5e4a-4344-8242-81646b958b75 None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:10.456064 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefcaec-5e4a-4344-8242-81646b958b75 None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:10.456379 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefcaec-5e4a-4344-8242-81646b958b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:10.476547 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-dbefcaec-5e4a-4344-8242-81646b958b75 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:53:10.477588 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dbefcaec-5e4a-4344-8242-81646b958b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:10.537240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1459/2886] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:53:10.547534 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:10.549407 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:10.616437 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f32a3b-e9ec-4ffc-9e4a-250627d19834 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:10.620032 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fe60c-f558-469b-a04a-38c69aada2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:10.620672 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe60c-f558-469b-a04a-38c69aada2eb service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:10.620956 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe60c-f558-469b-a04a-38c69aada2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:10.621237 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe60c-f558-469b-a04a-38c69aada2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:10.631529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fe60c-f558-469b-a04a-38c69aada2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:10.632995 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fe60c-f558-469b-a04a-38c69aada2eb service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:10.642732 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1428/2887] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:53:11.068144 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefcaec-5e4a-4344-8242-81646b958b75 None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:11.070119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-dbefcaec-5e4a-4344-8242-81646b958b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:11.085240 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefcaec-5e4a-4344-8242-81646b958b75 None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:11.090938 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:11.091586 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.091975 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.092304 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] PATH_INFO: `/v3/projects/cd45ad4a80ab4bf0a4a85e5271b13f8d` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.093455 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd45ad4a80ab4bf0a4a85e5271b13f8d)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:11.095424 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:11.229504 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd45ad4a80ab4bf0a4a85e5271b13f8d', 'request_id': 'req-4d6463b7-f628-4877-9982-11535853e5e7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:11.235609 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd45ad4a80ab4bf0a4a85e5271b13f8d', 'request_id': 'req-4d6463b7-f628-4877-9982-11535853e5e7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:11.302107 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd45ad4a80ab4bf0a4a85e5271b13f8d', 'request_id': 'req-4d6463b7-f628-4877-9982-11535853e5e7'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:11.313887 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] The token cache is being invalidate because project cd45ad4a80ab4bf0a4a85e5271b13f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:53:11.314175 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:11.314806 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-4d6463b7-f628-4877-9982-11535853e5e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:11.316320 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1460/2888] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:53:11 2022] DELETE /identity/v3/projects/cd45ad4a80ab4bf0a4a85e5271b13f8d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:53:11.322355 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db75bef-2532-4c59-a93a-edb386525675 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.322630 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db75bef-2532-4c59-a93a-edb386525675 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.322909 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db75bef-2532-4c59-a93a-edb386525675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.342700 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-6db75bef-2532-4c59-a93a-edb386525675 None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:53:11.343917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-6db75bef-2532-4c59-a93a-edb386525675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:11.361064 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:53:11.404827 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1429/2889] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:53:11.413291 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792add74-c2db-477a-9e88-7f00c6ef87be None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.413581 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792add74-c2db-477a-9e88-7f00c6ef87be None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.413865 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792add74-c2db-477a-9e88-7f00c6ef87be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.434082 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-792add74-c2db-477a-9e88-7f00c6ef87be None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:53:11.435217 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-792add74-c2db-477a-9e88-7f00c6ef87be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:11.443837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1461/2890] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:53:11.448549 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db75bef-2532-4c59-a93a-edb386525675 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:11.458994 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db75bef-2532-4c59-a93a-edb386525675 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:11.463583 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:11.464203 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.464509 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.464850 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] PATH_INFO: `/v3/domains` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.465428 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81685) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 13:53:11.465728 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:11.467314 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:11.478869 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1430/2891] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 13:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 13:53:11.483260 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792add74-c2db-477a-9e88-7f00c6ef87be None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:11.492854 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792add74-c2db-477a-9e88-7f00c6ef87be None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:11.497530 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5e03d-ba50-4796-ab8f-5a06d8e02639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:11.498125 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5e03d-ba50-4796-ab8f-5a06d8e02639 admin admin] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.498409 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5e03d-ba50-4796-ab8f-5a06d8e02639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.498756 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5e03d-ba50-4796-ab8f-5a06d8e02639 admin admin] PATH_INFO: `/v3/projects` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.499388 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5e03d-ba50-4796-ab8f-5a06d8e02639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:11.501337 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5e03d-ba50-4796-ab8f-5a06d8e02639 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:11.587969 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1462/2892] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 13:53:11 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:53:11.592427 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:11.602975 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09990829-4ae7-4786-9cc4-268010856994 admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:11.661222 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4a84f-0aad-46aa-8996-da8244c5dd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:11.661917 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4a84f-0aad-46aa-8996-da8244c5dd6a admin admin] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.662114 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4a84f-0aad-46aa-8996-da8244c5dd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.662385 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4a84f-0aad-46aa-8996-da8244c5dd6a admin admin] PATH_INFO: `/v3/users` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.663013 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4a84f-0aad-46aa-8996-da8244c5dd6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:11.664952 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4a84f-0aad-46aa-8996-da8244c5dd6a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:11.763713 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1431/2893] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 13:53:11 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 13:53:11.767967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5e03d-ba50-4796-ab8f-5a06d8e02639 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:11.778340 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5e03d-ba50-4796-ab8f-5a06d8e02639 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:11.783034 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d80069-34d0-49dc-bb97-d0e480da3437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:11.783695 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80069-34d0-49dc-bb97-d0e480da3437 admin admin] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.784052 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80069-34d0-49dc-bb97-d0e480da3437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.784367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80069-34d0-49dc-bb97-d0e480da3437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.785009 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d80069-34d0-49dc-bb97-d0e480da3437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:11.786627 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d80069-34d0-49dc-bb97-d0e480da3437 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:11.796845 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1463/2894] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 13:53:11.800894 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4a84f-0aad-46aa-8996-da8244c5dd6a admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:11.811446 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4a84f-0aad-46aa-8996-da8244c5dd6a admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:11.816078 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbb63-83e0-4f13-b86a-e74f0e84ce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:11.816705 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbb63-83e0-4f13-b86a-e74f0e84ce6a admin admin] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.817023 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbb63-83e0-4f13-b86a-e74f0e84ce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.817843 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbb63-83e0-4f13-b86a-e74f0e84ce6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.818485 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cbb63-83e0-4f13-b86a-e74f0e84ce6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:11.820439 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cbb63-83e0-4f13-b86a-e74f0e84ce6a admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:11.832103 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1432/2895] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 13:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 13:53:11.837837 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d80069-34d0-49dc-bb97-d0e480da3437 admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:11.848336 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d80069-34d0-49dc-bb97-d0e480da3437 admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:11.852998 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:11.853669 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:11.853954 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:11.854232 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] PATH_INFO: `/v3/projects/59c5aa64aa034a5da2a19b23df5e6028/users/87206b3f79cd4fb1b200f5e583448c01/roles/b4f974426b2b44d9843cb577d1b7e396` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:11.869696 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c5aa64aa034a5da2a19b23df5e6028, user_id=87206b3f79cd4fb1b200f5e583448c01, role_id=b4f974426b2b44d9843cb577d1b7e396)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:11.872363 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:11.993727 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1464/2896] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 13:53:11 2022] PUT /identity/v3/projects/59c5aa64aa034a5da2a19b23df5e6028/users/87206b3f79cd4fb1b200f5e583448c01/roles/b4f974426b2b44d9843cb577d1b7e396 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:53:12.009675 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30035b37-8c1d-4a7e-9df7-be4359a0e007 None None] REQUEST_METHOD: `POST` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:12.009904 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30035b37-8c1d-4a7e-9df7-be4359a0e007 None None] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:12.010184 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30035b37-8c1d-4a7e-9df7-be4359a0e007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:12.040028 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.auth.core [None req-30035b37-8c1d-4a7e-9df7-be4359a0e007 None None] MFA Rules not processed for user `87206b3f79cd4fb1b200f5e583448c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81685) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:53:12.040383 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-30035b37-8c1d-4a7e-9df7-be4359a0e007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:12.107152 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1433/2897] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:53:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 13:53:12.119218 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:12.121796 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:12.143464 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:53:12.183367 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fafce3-e6ec-43f4-ad04-e7c778e4cf4f admin admin] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:12.187257 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a0d97-b1bd-4d1d-a7f1-7715e029b066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2550e85fa53d416887bff88fbff937d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:12.187878 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0d97-b1bd-4d1d-a7f1-7715e029b066 service cinder] REQUEST_METHOD: `GET` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:12.188423 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0d97-b1bd-4d1d-a7f1-7715e029b066 service cinder] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:12.189195 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0d97-b1bd-4d1d-a7f1-7715e029b066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:12.198404 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a0d97-b1bd-4d1d-a7f1-7715e029b066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:12.199862 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a0d97-b1bd-4d1d-a7f1-7715e029b066 service cinder] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:12.210626 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1465/2898] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:53:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:53:22.103852 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:53:22.244279 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=17 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:53:51.095536 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30035b37-8c1d-4a7e-9df7-be4359a0e007 None None] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:51.157225 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30035b37-8c1d-4a7e-9df7-be4359a0e007 None None] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:51.162050 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:51.162673 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:51.162957 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:51.163231 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] PATH_INFO: `/v3/users/87206b3f79cd4fb1b200f5e583448c01` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:51.164976 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=87206b3f79cd4fb1b200f5e583448c01)` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:51.167247 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:51.333130 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87206b3f79cd4fb1b200f5e583448c01'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:51.333746 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87206b3f79cd4fb1b200f5e583448c01'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:51.341960 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87206b3f79cd4fb1b200f5e583448c01'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:51.342608 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87206b3f79cd4fb1b200f5e583448c01'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:51.398395 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.notifications [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87206b3f79cd4fb1b200f5e583448c01'} {{(pid=81685) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:51.417704 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1434/2899] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 13:53:51 2022] DELETE /identity/v3/users/87206b3f79cd4fb1b200f5e583448c01 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 13:53:51.422406 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e62d84-80dd-430a-a056-2f978a437c2c None None] REQUEST_METHOD: `POST` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:51.426790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e62d84-80dd-430a-a056-2f978a437c2c None None] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:51.426790 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e62d84-80dd-430a-a056-2f978a437c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:51.443136 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.auth.core [None req-86e62d84-80dd-430a-a056-2f978a437c2c None None] MFA Rules not processed for user `23e13f53885b4bcd974e51d34ba1099e`. Rule list: `[]` (Enabled: `True`). {{(pid=81686) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 13:53:51.444353 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-86e62d84-80dd-430a-a056-2f978a437c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:51.507384 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1466/2900] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 13:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 13:53:51.517779 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:51.519209 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:51.578604 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de137c-321f-47ab-8bb2-515e343bd8be admin admin] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:51.582365 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18ca375f53414475b75dae308cd19fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:51.582967 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:51.583302 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:51.583529 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:51.593423 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:51.594865 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:51.605041 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1435/2901] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:53:51.994864 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e62d84-80dd-430a-a056-2f978a437c2c None None] Authenticating user token {{(pid=81686) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:53:51.999949 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.fernet_utils [None req-86e62d84-80dd-430a-a056-2f978a437c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81686) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:53:52.009505 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e62d84-80dd-430a-a056-2f978a437c2c None None] Validating token access rules against request {{(pid=81686) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:53:52.014331 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '23e13f53885b4bcd974e51d34ba1099e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e28975466b54f229ea6496a2befdfd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81686) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:53:52.014967 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:53:52.015248 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:53:52.015526 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] PATH_INFO: `/v3/projects/59c5aa64aa034a5da2a19b23df5e6028` {{(pid=81686) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:53:52.016683 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c5aa64aa034a5da2a19b23df5e6028)` {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:53:52.018590 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] RBAC: Authorization granted {{(pid=81686) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:53:52.153996 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c5aa64aa034a5da2a19b23df5e6028', 'request_id': 'req-58ef7fd4-7c97-42be-b2b5-ed8e63880789'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:52.159667 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c5aa64aa034a5da2a19b23df5e6028', 'request_id': 'req-58ef7fd4-7c97-42be-b2b5-ed8e63880789'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:52.261119 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c5aa64aa034a5da2a19b23df5e6028', 'request_id': 'req-58ef7fd4-7c97-42be-b2b5-ed8e63880789'} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:52.273001 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] The token cache is being invalidate because project 59c5aa64aa034a5da2a19b23df5e6028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81686) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 13:53:52.273285 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:52.274101 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG keystone.notifications [None req-58ef7fd4-7c97-42be-b2b5-ed8e63880789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81686) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 13:53:52.275906 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: [pid: 81686|app: 0|req: 1467/2902] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 13:53:51 2022] DELETE /identity/v3/projects/59c5aa64aa034a5da2a19b23df5e6028 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 13:54:01.600257 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:54:02.270314 dv-cinder-779578-4-iscsi devstack@keystone.service[81686]: DEBUG dbcounter [-] [81686] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81686) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 13:57:37.092547 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] Authenticating user token {{(pid=81685) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 13:57:37.093999 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:57:37.234227 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd633b5-57a4-4137-ad96-5b021fc2ca3a service neutron] Validating token access rules against request {{(pid=81685) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 13:57:37.292101 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a763140-baab-4aca-8ec8-cfdffc2355bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29f79ed2af4341ab6f5ba50a47d8bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2c71d386314c7190d1f2e2e3038636', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81685) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 13:57:37.292734 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a763140-baab-4aca-8ec8-cfdffc2355bb service placement] REQUEST_METHOD: `GET` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 13:57:37.293015 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a763140-baab-4aca-8ec8-cfdffc2355bb service placement] SCRIPT_NAME: `/identity` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 13:57:37.293289 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a763140-baab-4aca-8ec8-cfdffc2355bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81685) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 13:57:37.295835 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.fernet_utils [None req-3a763140-baab-4aca-8ec8-cfdffc2355bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81685) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 13:57:37.306972 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a763140-baab-4aca-8ec8-cfdffc2355bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 13:57:37.308574 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a763140-baab-4aca-8ec8-cfdffc2355bb service placement] RBAC: Authorization granted {{(pid=81685) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 13:57:37.318764 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: [pid: 81685|app: 0|req: 1436/2903] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 13:57:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 13:57:47.314255 dv-cinder-779578-4-iscsi devstack@keystone.service[81685]: DEBUG dbcounter [-] [81685] Writing DB stats keystone:SELECT=89 {{(pid=81685) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}